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Preface (Part I) 



This book, Part I - Basic Linear Algebra, covers Chapters through 5 of the book A Compre- 
hensive Introduction to Linear Algebra (Addison- Wesley, 1986), by Joel G. Broida and S. Gill 
Williamson. Chapters and 1 arc for review as needed. Chapters 2 through 5, supplemented 
by selections from Part II and Part III in this series, is suitable for a first course in linear 
algebra for upper division undergraduates. The original Preface (included here) gives other 
suggestions. 
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Preface (Parts I, II, III) 



As a text, this book is intended for upper division undergraduate and begin- 
ning graduate students in mathematics, applied mathematics, and fields of 
science and engineering that rely heavily on mathematical methods. However, 
it has been organized with particular concern for workers in these diverse 
fields who want to review the subject of linear algebra. In other words, we 
have written a book which we hope will still be referred to long after any final 
exam is over. As a result, we have included far more material than can possi- 
bly be covered in a single semester or quarter. This accomplishes at least two 
things. First, it provides the basis for a wide range of possible courses that can 
be tailored to the needs of the student or the desire of the instructor. And 
second, it becomes much easier for the student to later learn the basics of 
several more advanced topics such as tensors and infinite-dimensional vector 
spaces from a point of view coherent with elementary linear algebra. Indeed, 
we hope that this text will be quite useful for self-study. Because of this, our 
proofs are extremely detailed and should allow the instructor extra time to 
work out exercises and provide additional examples if desired. 
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PREFACE 



A major concern in writing this book has been to develop a text that 
addresses the exceptional diversity of the audience that needs to know some- 
thing about the subject of linear algebra. Although seldom explicitly 
acknowledged, one of the central difficulties in teaching a linear algebra 
course to advanced students is that they have been exposed to the basic back- 
ground material from many different sources and points of view. An experi- 
enced mathematician will see the essential equivalence of these points of 
view, but these same differences seem large and very formidable to the 
students. An engineering student for example, can waste an inordinate amount 
of time because of some trivial mathematical concept missing from then- 
background. A mathematics student might have had a concept from a different 
point of view and not realize the equivalence of that point of view to the one 
currently required. Although such problems can arise in any advanced mathe- 
matics course, they seem to be particularly acute in linear algebra. 

To address this problem of student diversity, we have written a very self- 
contained text by including a large amount of background material necessary 
for a more advanced understanding of linear algebra. The most elementary of 
this material constitutes Chapter 0, and some basic analysis is presented in 
three appendices. In addition, we present a thorough introduction to those 
aspects of abstract algebra, including groups, rings, fields and polynomials 
over fields, that relate directly to linear algebra. This material includes both 
points that may seem "trivial" as well as more advanced background material. 
While trivial points can be quickly skipped by the reader who knows them 
already, they can cause discouraging delays for some students if omitted. It is 
for this reason that we have tried to err on the side of over-explaining 
concepts, especially when these concepts appear in slightly altered forms. The 
more advanced reader can gloss over these details, but they are there for those 
who need them. We hope that more experienced mathematicians will forgive 
our repetitive justification of numerous facts throughout the text. 

A glance at the Contents shows that we have covered those topics nor- 
mally included in any linear algebra text although, as explained above, to a 
greater level of detail than other books. Where we differ significantly in con- 
tent from most linear algebra texts however, is in our treatment of canonical 
forms (Chapter 8), tensors (Chapter 11), and infinite-dimensional vector 
spaces (Chapter 12). In particular, our treatment of the Jordan and rational 
canonical forms in Chapter 8 is based entirely on invariant factors and the 
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Smith normal form of a matrix. We feel this approach is well worth the effort 
required to learn it since the result is, at least conceptually, a constructive 
algorithm for computing the Jordan and rational forms of a matrix. However, 
later sections of the chapter tie together this approach with the more standard 
treatment in terms of cyclic subspaces. Chapter 1 1 presents the basic formal- 
ism of tensors as they are most commonly used by applied mathematicians, 
physicists and engineers. While most students first learn this material in a 
course on differential geometry, it is clear that virtually all the theory can be 
easily presented at this level, and the extension to differentiable manifolds 
then becomes only a technical exercise. Since this approach is all that most 
scientists ever need, we leave more general treatments to advanced courses on 
abstract algebra. Finally, Chapter 12 serves as an introduction to the theory of 
infinite-dimensional vector spaces. We felt it is desirable to give the student 
some idea of the problems associated with infinite-dimensional spaces and 
how they are to be handled. And in addition, physics students and others 
studying quantum mechanics should have some understanding of how linear 
operators and their adjoints are properly defined in a Hilbert space. 

One major topic we have not treated at all is that of numerical methods. 
The main reason for this (other than that the book would have become too 
unwieldy) is that we feel at this level, the student who needs to know such 
techniques usually takes a separate course devoted entirely to the subject of 
numerical analysis. However, as a natural supplement to the present text, we 
suggest the very readable "Numerical Analysis" by I. Jacques and C. Judd 
(Chapman and Hall, 1987). 

The problems in this text have been accumulated over 25 years of teaching 
the subject of linear algebra. The more of these problems that the students 
work the better. Be particularly wary of the attitude that assumes that some of 
these problems are "obvious" and need not be written out or precisely articu- 
lated. There are many surprises in the problems that will be missed from this 
approach! While these exercises are of varying degrees of difficulty, we have 
not distinguished any as being particularly difficult. However, the level of dif- 
ficulty ranges from routine calculations that everyone reading this book 
should be able to complete, to some that will require a fair amount of thought 
from most students. 

Because of the wide range of backgrounds, interests and goals of both 
students and instructors, there is little point in our recommending a particular 
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course outline based on this book. We prefer instead to leave it up to each 
teacher individually to decide exactly what material should be covered to meet 
the needs of the students. While at least portions of the first seven chapters 
should be read in order, the remaining chapters are essentially independent of 
each other. Those sections that are essentially applications of previous 
concepts, or else are not necessary for the rest of the book are denoted by an 
asterisk (*). 

Now for one last comment on our notation. We use the symbol ■ to denote 
the end of a proof, and / to denote the end of an example. Sections are labeled 
in the format "Chapter.Section," and exercises are labeled in the format 
"Chapter.Section.Exercise." For example. Exercise 2.3.4 refers to Exercise 4 
of Section 2.3, i.e.. Section 3 of Chapter 2. Books listed in the bibliography 
are referred to by author and copyright date. 
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CHAPTER 



Foundations 



This text discusses the theory of finite-dimensional vector spaces in sufficient 
detail to enable the reader to understand and solve most linear algebra prob- 
lems in mathematics and physics likely to be encountered outside of special- 
ized research. In other words, we treat the general theory of determinants and 
matrices along with their relationship to linear transformations. Our approach 
will generally be rather abstract since we feel that most readers already have a 
reasonable amount of experience in visualizing vectors in three dimensions. 
Furthermore, we will not discuss any analytic geometry. Those readers who 
wish to learn something about this subject are referred to the books listed in 
the bibliography. 

In this chapter, we briefly go through some elementary concepts from 
analysis dealing with numbers and functions. While most readers will proba- 
bly be familiar with this material, it is worth summarizing the basic definitions 
that we will be using throughout this text, and thus ensure that everyone is on 
an equal footing to begin with. This has the additional advantage in that it also 
makes this text virtually self-contained and all the more useful for self-study. 
The reader should feel free to skim this chapter now, and return to certain 
sections if and when the need arises. 
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FOUNDATIONS 



0.1 SETS 

For our purposes, it suffices to assume that the concept of a set is intuitively 
clear, as is the notion of the set of integers. In other words, a set is a collection 
of objects, each of which is called a point or an element of the set. For exam- 
ple, the set of integers consists of the numbers 0, +1, +2, . . . and will be 
denoted by Z. Furthermore, the set Z"^ consisting of the numbers 1,2,... will 
be called the set of positive integers, while the collection 0, 1, 2, ... is called 
the set of natural numbers (or nonnegative integers). If m and n ^ are 
integers, then the set of all numbers of the form m/n is called the set of 
rational numbers, and will be denoted by Q. We shall shortly show that there 
exist real numbers not of this form. The most important sets of numbers that 
we shall be concerned with are the set IR of real numbers and the set C of 
complex numbers (both of these sets will be discussed below). 

If S and T are sets, then S is said to be a subset of T if every element of S 
is also an element of T, i.e., x G S implies x G T. If in addition S T, then S is 
said to be a proper subset of T. To denote the fact that S is a subset of T, we 
write S C T (or sometimes T D S in which case T is said to be a superset of 
S). Note that if S C T and T C S, then S = T. This fact will be extremely 
useful in many proofs. The set containing no elements at all is called the 
empty set and will be denoted by 0. 

Next, consider the set of all elements which are members of T but not 
members of S. This defines the set denoted by T - S and called the comple- 
ment of S in T. (Many authors denote this set by T \ S, but we shall not use 
this notation.) In other words, x G T - S means that x G T but x ^ S. If (as is 
usually the case) the set T is understood and S C T, then we write the com- 
plement of S as S'^. 

Example 0.1 Let us prove the useful fact that if A, B C X with A'^ C B, then 
it is true that B'^ C A. To show this, we simply note that x G B'^ implies x^ B, 
which then implies x ^ A'^, and hence x G A. This observation is quite useful 
in proving many identities involving sets. / 

Now let Si, S2, ... be a collection of sets. (Such a collection is called a 
family of sets.) For simplicity we write this collection as {Si}, i G I. The set I 

is called an index set, and is most frequently taken to be the set Z"^. The 
union Uiei Si of the collection {Si} is the set of all elements that are members 
of at least one of the Si. Since the index set is usually understood, we will 
simply write this as USi. In other words, we write 

USi = {x: x G Si for at least one i G 1} . 
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This notation will be used throughout this text, and is to be read as "the set of 
all X such that x is an element of Si for at least one i £ I." Similarly, the inter- 
section nSi of the Si is given by 

nSi = {x:xESiforalliei} . 

For example, if S, T C X, then S - T = S n T'= where T'= = X - T. 
Furthermore, two sets Si and S2 are said to be disjoint if Si fl S2 = 0. 

We now use these concepts to prove the extremely useful "De Morgan 
Formulas." 

Theorem 0.1 Let {Si} be a family of subsets of some set T. Then 

(a) USi'^ = (nSi)" 

(b) nSi'= = (uSi)'= 

Proof (a) X E: USi'^ if and only if x is an element of some Si'^, hence if and 
only if X is not an element of some Si, hence if and only if x is not an element 

of nSi, and therefore if and only if x e (flSi)'^. 

(b) X E nSi'^ if and only if x is an element of every Si'^, hence if and only 

if X is not an element of any Si, and therefore if and only if x £ (USi)'^. I 

While this may seem like a rather technical result, it is in fact directly 
useful not only in mathematics, but also in many engineering fields such as 
digital electronics where it may be used to simplify various logic circuits. 

Finally, if Si, S2, . . . , Sn is a collection of sets, we may form the (ordered) 
set of all n-tuples (xi , . . . , Xn) where each Xi £ Si. This very useful set is 
denoted by Si x • • • x Sn and called the Cartesian product of the Si. 

Example 0.2 Probably the most common example of the Cartesian product is 
the plane = R x R. Each point x G R^ has coordinates (x, y) where x, y G 
R. In order to facilitate the generalization to R", we will generally write x = 
(xi, X2) or X = (x\ x^). This latter notation is used extensively in more 
advanced topics such as tensor analysis, and there is usually no confusion 
between writing the components of x as superscripts and their being inter- 
preted as exponents (see Chapter 11). / 



4 



FOUNDATIONS 



Exercises 

1. Let A, B and C be sets. Prove that 

(a) (A - B) n C = (A n C) - (B n C). 

(b) A n (B U C) = (A n B) U (A n C). 

(c) A U (B n C) = (A U B) n (A U C). 

(d) (A - B) - C = A - (B U C). 

(e) A - (B U C) = (A - B) n (A - C). 

2. The symmetric difference A A B of two sets A and B is defined by 

A A B = (A - B) U (B - A). 

Show that 

(a) A A B = (A U B) - (A n B) = B A A. 

(b) A n (B A C) = (A n B) A (A n C). 

(c) A U B = (A A B) A (A n B). 

(d) A - B = A A (A n B). 

3. Let ^ be a nonempty collection of sets with the property that A, B £ ^ 
implies that both ARB and A A B are also in ^ . Show that ^ must con- 
tain the empty set, A U B and A - B. (The collection ^is called a ring of 
sets, and is of fundamental importance in measure theory and Lebesgue 
integration.) 

0.2 MAPPINGS 

Given two sets S and T, a mapping or function f from S to T is a rule which 
assigns a unique element y £ T to each element x £ S. Symbolically, we write 
this mapping asf:S^Torf:xi-^ f(x) (this use of the colon should not be 
confused with its usage meaning "such that"). The set S is called the domain 
of f and T is called the range of f. Each point f(x) G T is called the image of x 
under f (or the value of f at x), and the collection {f(x) G T: x G S} of all such 
image points is called the image of f. In general, whenever a new mapping is 
given, we must check to see that it is in fact well-defined. In other words, we 
must verify that x = y implies f(x) = f(y). We will use this requirement several 
times throughout the text. 

If A C S, the set {f(x): x G A} is called the image of A under f and is 
denoted by f(A). If f is a mapping from S to T and ACS, then the restriction 
of f to A, denoted by f|A (or sometimes fA), is the function from A to T 
defined by f|A: x G A f(x) G T. If x' G T, then any element x G S such that 
f(x) = x' is called an inverse image of x' (this is sometimes also called a 
preimage of x'). Note that in general there may be more than one inverse 
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image for any particular x' £ T. Similarly, if A' C T, then the inverse image of 
A' is the subset of S given by {x £ S: f(x) £ A'}. We will denote the inverse 
image of A' by f"'(A'). 

Let f be a mapping from S to T. Note that every element of T need not 
necessarily be the image of some element of S. However, if f(S) = T, then f is 
said to be onto or surjective. In other words, f is surjective if given any x' £ 
T there exists x e S such that f(x) = x'. In addition, f is said to be one-to-one 
or injective ifx^y implies that f(x) ^ f(y). An alternative characterization is 
to say that f is injective if f(x) = f(y) implies that x = y. 

If f is both injective and surjective, then f is said to be bijective. In this 
case, given any x' £ T there exists a unique x £ S such that x' = f(x). If f is 
bijective, then we may define the inverse mapping f : T ^ S in the follow- 
ing way. For any x' £ T, we let f"'(x') be that (unique) element x £ S such that 
f(x) =x'. 

Example 0.3 Consider the function f: R R defined by f(x) = x^. This 
mapping is clearly not surjective since f(x) > for any x G R. Furthermore, it 
is also not injective. Indeed, it is clear that 2^-2 but f(2) = f(-2) = 4. Note 
also that both the domain and range of f are the whole set R, but that the 
image of f is just the subset of all nonnegative real numbers (i.e., the set of all 
xeRwithx >0). 

On the other hand, it is easy to see that the mapping g: R — * R defined by 
g(x) = ax + b for any a, b G R (with a ^ 0) is a bijection. In this case the 
inverse mapping is simply given by g"' (x') = (x' - b)/a. / 

Example 0.4 If f is a mapping defined on the collections {Aj} and {Bj} of 
sets, then we claim that 

f(UAO = Uf(AO 

and 

f-'(UBi) = Uf-^Bi) . 

To prove these relationships we proceed in our usual manner. Thus we have 
x' G f(UAi) if and only if x' = f(x) for some x G UAj, hence if and only if x' is 
in some f(Ai), and therefore if and only if x' G Uf(Ai). This proves the first 
statement. As to the second statement, we have x G f"'(UBi) if and only if 
f(x) G UBi, hence if and only if f(x) is in some Bj, hence if and only if x is in 
some f~\Bi), and therefore if and only if x G Uf"'(Bi). 

Several similar relationships that will be referred to again are given in the 
exercises. / 
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Now consider the sets S, T and U along with the mappings f: S ^ T and 
g: T U. We define the composite mapping (sometimes also called the 
product) g ° f: S ^ U by 

(g o f)(x) = g(f(x)) 

for all X £ S. In general, f ° g ^ g ° f , and we say that the composition of two 
functions is not commutative. However, if we also have a mapping h: U ^ 
V, then for any x £ S we have 

{ho {go f)){x) = h{{g o f){x)) = h{g{f{x))) = {h o g){f{x)) 

= ((hog)of){x) 

This means that 

h o (g o f) = (h o g) o f 

and hence the composition of mappings is associative. 

As a particular case of the composition of mappings, note that if f: S T 
is a bijection and f(x) = x' £ T where x £ S, then 

(fof-i)(x') = f(f-'(x')) = f(x) = x' 

and 

(f-' ° f)(x) = f-'(f(x)) = f-^x') = X . 

If we write f o f = Ij, then the mapping It has the property that It(x') = x' 
for every x' £ T. We call It the identity mapping on T. Similarly, the 
composition mapping f"' » f = Is is called the identity mapping on S. In the 
particular case that S = T, then f » f"' = f"^ » f = I is also called the identity 
mapping. 

An extremely important result follows by noting that (even if S 9^ T) 

if-' ° g-'xg ° f){x) = (/"' ° g-'){g(f{x))) = f-'{g-'{g{f{xm 
=r\f{x))=x 

Since it is also easy to see that (g ° f)(f"' ° g"')(x') = x', we have shown that 



(gof)-I = f-Iog-I 
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Exercises 

1. Let f be a mapping of sets. For each of the following, state any conditions 
on f that may be required (e.g., surjective or injective), and then prove the 
statement: 

(a) Ai C A2 implies f(Ai) C fCAj). 

(b) f(A)'^ C f(A'^) is true if and only if f is surjective. 

(c) f(nA,) c nf(A,). 

(d) Bi C B2 implies f-'(Bi) C f-'(B2). 

(e) f-'(nBO = nf-'(BO. 

(f) f-HB^) = f-HB)'=. 

2. Given a nonempty set A, we define the identity mapping Ia: A ^ A by 

iA(a) = a for every a G A. Let f: A ^ A be any mapping. 

(a) Show that f ° Ia = Ia ° f = f. 

(b) If f is bijective (so that f ' exists), show that f ° f ' = f ' ° f = Ia • 

(c) Let f be a bijection, and suppose that g is any other mapping with the 
property that g°f=f °g = iA. Show that g = f"' . 

0.3 ORDERINGS AND EQUIVALENCE RELATIONS 

Given any two sets S and T, a subset R of S x T is said to be a relation 
between S and T. If R C S x T and (x, y) G R, then it is common to write xRy 
to show that x and y are "R-related." In particular, consider the relation sym- 
bolized by < and defined as having the following properties on a set S: 

(a) X < X (reflexivity); 

(b) X < y and y < x implies x = y for all x, y G S (antisymmetry); 

(c) X < y and y < z implies x < z for all x, y, z G S (transitivity). 

Any relation on a non-empty set S having these three properties is said to 
be a partial ordering, and S is said to be a partially ordered set. We will 

sometimes write y > x instead of the equivalent notation x < y. The reason for 
including the qualifying term "partial" in this definition is shown in our next 
example. 

Example 0.5 Let S be any set, and let !P(S) be the collection of all subsets of 
S (this is sometimes called the power set of S). If A, B and C are subsets of S, 
then clearly A C A so that (a) is satisfied; A C B and B C A implies A = B 
then satisfies (b); and A C B and B C C implies A C C satisfies (c). Therefore 
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C defines a partial ordering on !P(S), and the subsets of S are said to be 
ordered by inclusion. Note however, that if A C S and B C S but A(tB and 
B ^ A, then there is no relation between A and B, and we say that A and B are 
not comparable. / 

The terminology used in this example is easily generalized as follows. If S 
is any partially ordered set and x, y £ S, then we say that x and y are compa- 
rable if either x < y or y < x. 

If, in addition to properties (a) - (c), a relation R also has the property that 
any two elements are comparable, then R is said to be a total ordering. In 
other words, a total ordering also has the property that 

(d) either x < y or y < x for all x, y £ S. 

Let S be a set partially ordered by < and suppose A C S. It should be clear 
that A may be considered to be a partially ordered set by defining a < b for all 
a, b E A if a < b where a and b are considered to be elements of S. (This is 
similar to the restriction of a mapping.) We then say that A has a partial 
ordering < induced by the ordering on S. If A is totally ordered by the order- 
ing induced by <, then A is frequently called a chain in S. 

Let A be a non-empty subset of a partially ordered set S. An element 
X G S is called an upper bound for A if a < xfor all a G A. If it so happens 
that X is an element of A, then x is said to be a largest element of A. 
Similarly, y G S is called a lower bound for A if y < a for all a G A, and y is a 
smallest element of A if y G A. If A has an upper (lower) bound, then we say 
that A is bounded above (below). Note that largest and smallest elements 
need not be unique. 

Suppose that A is bounded above by a G S, and in addition, suppose that 
for any other upper bound x of A we have a < x. Then we say that a is a least 
upper bound (or supremum) of A, and we write a = lub A = sup A. As 
expected, if A is bounded below by (3 G S, and if y < P for all other lower 
bounds y G S, then |3 is called a greatest lower bound (or inflmum), and we 
write p = gib A = inf A. In other words, if it exists, the least upper (greatest 
lower) bound for A is a smallest (largest) element of the set of all upper 
(lower) bounds for A. 

From property (b) above and the definitions of inf and sup, we see that if 
they exist, the least upper bound and the greatest lower bound are unique. (For 
example, if p and p' are both greatest lower bounds, then p < p' and P' ^ P 
implies that p = P'.) Hence it is meaningful to talk about the least upper 
bound and the greatest lower bound. 

Let S be a partially ordered set, and suppose ACS. An element a G A is 
said to be maximal in A if for any element a G A with a < a, we have a = a. 
In other words, no element of A other than a itself is greater than or equal to 
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a. Similarly, an element P £ A is said to be minimal in A if for any b £ A 
with b < (3, we have b = (3. Note that a maximal element may not be a largest 
element (since two elements of a partially ordered set need not be 
comparable), and there may be many maximal elements in A. 

We now state Zom's lemma, one of the most fundamental results in set 
theory, and hence in all of mathematics. While the reader can hardly be 
expected to appreciate the significance of this lemma at the present time, it is 
in fact extremely powerful. 

Zorn's Lemma Let S be a partially ordered set in which every chain has an 
upper bound. Then S contains a maximal element. 

It can be shown (see any book on set theory) that Zom's lemma is logi- 
cally equivalent to the axiom of choice, which states that given any non- 
empty family of non-empty disjoint sets, a set can be formed which contains 
precisely one element taken from each set in the family. Although this seems 
like a rather obvious statement, it is important to realize that either the axiom 
of choice or some other statement equivalent to it must be postulated in the 
formulation of the theory of sets, and thus Zom's lemma is not really provable 
in the usual sense. In other words, Zorn's lemma is frequently taken as an 
axiom of set theory. However, it is an indispensable part of some of what fol- 
lows although we shall have little occasion to refer to it directly. 

Up to this point, we have only talked about one type of relation, the partial 
ordering. We now consider another equally important relation. Let S be any 
set. A relation a on S is said to be an equivalence relation if it has the fol- 
lowing properties for all x, y, z £ S: 

(a) X = X for all x G S (reflexivity); 

(b) X s: y implies y = x (symmetry); 

(c) X = y and y = z implies x = z for all x, y, z G S (transitivity). 

Note that only (b) differs from the defining relations for a partial ordering. 

A partition of a set S is a family {Si} of non-empty subsets of S such that 
USi = S and Si H Sj ^ implies Si = Sj. Suppose x G S and let = be an 
equivalence relation on S. The subset of S defined by [x] = {y: y = x} is called 
the equivalence class of x. The most important property of equivalence rela- 
tions is contained in the following theorem. 
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Theorem 0.2 The family of all distinct equivalence classes of a set S forms a 
partition of S. (This is called the partition induced by .) Moreover, given any 
partition of S, there is an equivalence relation on S that induces this partition. 

Proof Let a be an equivalence relation on a set S, and let x be any element 
of S. Since x = x, it is obvious that x £ [x]. Thus each element of S lies in at 
least one non-empty equivalence class. We now show that any two equiv- 
alence classes are either disjoint or are identical. Let [xj and [xj] be two 
equivalence classes, and let y be a member of both classes. In other words, y = 
Xi and y = X2. Now choose any z G [xj so that z Xi. But this means that z =: 
Xi = y s: X2 so that any element of [xj is also an element of [xz], and hence 
[xi] C [xj]. Had we chosen z G [xj] we would have found that [xj] C [xj. 
Therefore [xj = [X2], and we have shown that if two equivalence classes have 
any element in common, then they must in fact be identical. 

Let {Si} be any partition of S. We define an equivalence relation on S by 
letting X = y if X, y E Si for any x, y E S. It should be clear that this does 
indeed satisfy the three conditions for an equivalence relation, and that this 
equivalence relation induces the partition {Si}. I 

As we will see in the next chapter, this theorem has a direct analogue in 
the theory of groups. 

Exercises 

1. Let Z denote the set of positive integers. We write m|n to denote the fact 
that m divides n, i.e., n = km for some k E Z 

(a) Show that | defines a partial ordering on Z 

(b) Does Z ^ contain either a maximal or minimal element relative to this 
partial ordering? 

(c) Prove that any subset of Z"^ containing exactly two elements has a 
greatest lower bound and a least upper bound. 

(d) For each of the following subsets of Z determine whether or not it is 
a chain in Z *, find a maximal and minimal element, an upper and lower 
bound, and a least upper bound: 

(i) {1,2,4,6,8}. 

(ii) {1,2,3,4,5}. 

(iii) {3, 6, 9, 12, 15, 18}. 

(iv) {4, 8, 16, 32, 64, 128}. 
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2. Define a relation a on IR by requiring that a ~ b if |a| = |b|. Show that this 
defines an equivalence relation on R. 

3. For any a, b G R, let a ~ b mean ab > 0. Does ~ define an equivalence rela- 
tion? What happens if we use ab > instead of ab > 0? 

0.4 CARDINALITY AND THE REAL NUMBER SYSTEM 

We all have an intuitive sense of what it means to say that two finite sets have 
the same number of elements, but our intuition leads us astray when we come 
to consider infinite sets. For example, there are as many perfect squares (1, 4, 
9, 16, etc.) among the positive integers as there are positive integers. That this 
is true can easily be seen by writing each positive integer paired with its 
square: 

1, 2, 3, 4, ... 

i', 2\ 3^ 4^ ... 

While it seems that the perfect squares are only sparsely placed throughout the 
integers, we have in fact constructed a bijection of all positive integers with all 
of the perfect squares of integers, and we are forced to conclude that in this 
sense they both have the "same number of elements." 

In general, two sets S and T are said to have the same cardinality, or to 
possess the same number of elements, if there exists a bijection from S to T. A 
set S is finite if it has the same cardinality as either or the set {1, 2, ... , n} 
for some positive integer n; otherwise, S is said to be infinite. However, there 
are varying degrees of "infinity." A set S is countable if it has the same car- 
dinality as a subset of the set Z of positive integers. If this is not the case, 
then we say that S is uncountable. Any infinite set which is numerically 
equivalent to (i.e., has the same cardinality as) Z"^ is said to be countably 
infinite. We therefore say that a set is countable if it is countably infinite or if 
it is non-empty and finite. 

It is somewhat surprising (as was first discovered by Cantor) that the set 

Q"^ of all positive rational numbers is in fact countable. The elements of Q"^ 
can not be listed in order of increasing size because there is no smallest such 
number, and between any two rational numbers there are infinitely many 

others (see Theorem 0.4 below). To show that Q"^ is countable, we shall con- 
struct a bijection from Z to Q"^ . 

To do this, we first consider all positive rationals whose numerator and 
denominator add up to 2. In this case we have only 1/1 = 1. Next we list those 
positive rationals whose numerator and denominator add up to 3. If we agree 
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to always list our rationals with numerators in increasing order, then we have 
1/2 and 2/1 = 2. Those rationals whose numerator and denominator add up to 
4 are then given by 1/3, 2/2 = 1, 3/1 = 3. Going on to 5 we obtain 1/4, 2/3, 3/2, 
4/1 = 4. For 6 we have 1/5, 2/4 = 1/2, 3/3 = 1, 4/2 = 2, 5/1 = 5. Continuing 
with this procedure, we list together all of our rationals, omitting any number 
already listed. This gives us the sequence 

1, 1/2, 2, 1/3, 3, 1/4, 2/3, 3/2, 4, 1/5, 5, . . . 

which contains each positive rational number exactly once, and provides our 
desired bijection. 

We have constructed several countably infinite sets of real numbers, and it 
is natural to wonder whether there are in fact any uncountably infinite sets. It 
was another of Cantor's discoveries that the set IR of all real numbers is actu- 
ally uncountable. To prove this, let us assume that we have listed (in some 

manner similar to that used for the set Q"^) all the real numbers in decimal 
form. What we shall do is construct a decimal .did2d3- • • that is not on our list, 
thus showing that the list can not be complete. Consider only the portion of 
the numbers on our list to the right of the decimal point, and look at the first 
number on the list. If the first digit after the decimal point of the first number 
is a 1, we let di = 2; otherwise we let di =1. No matter how we choose the 
remaining d's, our number will be different from the first on our list. Now 
look at the second digit after the decimal point of the second number on our 
list. Again, if this second digit is a 1, we let dx = 2; otherwise we let d2 = 1. 
We have now constructed a number that differs from the first two numbers on 
our list. Continuing in this manner, we construct a decimal .did2d3- • • that 
differs from every other number on our list, contradicting the assumption that 
all real numbers can be listed, and proving that IR is actually uncountable. 

Since it follows from what we showed above that the set Q of all rational 
numbers on the real line is countable, and since we just proved that the set IR 
is uncountable, it follows that a set of irrational numbers must exist and be 
uncountably infinite. 

From now on we will assume that the reader understands what is meant by 
the real number system, and we proceed to investigate some of its most useful 
properties. A complete axiomatic treatment that justifies what we already 
know would take us too far afield, and the interested reader is referred to, e.g., 
Rudin (1976). 

Let S be any ordered set, and let A C S be non-empty and bounded above. 
We say that S has the least upper bound property if sup A exists in S. In the 
special case of S = IR, we have the following extremely important axiom. 



0.4 CARDINALITY AND THE REAL NUMBER SYSTEM 



13 



Archimedean Axiom Every non-empty set of real numbers which has an 
upper (lower) bound has a least upper bound (greatest lower bound). 

The usefulness of this axiom is demonstrated in the next rather obvious 
though important result, sometimes called the Archimedean property of the 
real number line. 

Theorem 0.3 Let a, b £ IR and suppose a > 0. Then there exists n £ Z such 
that na > b. 

Proof Let S be the set of all real numbers of the form na where n is a positive 
integer. If the theorem were false, then b would be an upper bound for S. But 
by the Archimedean axiom, S has a least upper bound a = sup S. Since a > 0, 
we have a - a < a and a - a can not be an upper bound of S (by definition of 
a). Therefore, there exists an m G Z such that ma G S and a - a < ma. But 
then a < (m + l)a G S which contradicts the fact that a = sup S. I 

One of the most useful facts about the real line is that the set Q of all 
rational numbers is dense in R. By this we mean that given any two distinct 
real numbers, we can always find a rational number between them. This 
means that any real number may be approximated to an arbitrary degree of 
accuracy by a rational number. It is worth proving this using Theorem 0.3. 

Theorem 0.4 Suppose x, y G IR and assume that x < y. Then there exists a 
rational number p G Q such that x < p < y. 

Proof Since x < y we have y - x > 0. In Theorem 0.3, choose a = y - x and 
b = 1 so there exists n G Z such that n(y - x) > 1, or alternatively, 

1 + nx < ny . 

Applying Theorem 0.3 again, we let a = 1 and both b = nx and b = -nx to find 

integers mi, mj G Z such that mi > nx and mj > -nx. Rewriting the second of 
these as -mj < nx, we combine the two inequalities to obtain 

-m2 < nx < mi 

so that nx lies between two integers. But if nx lies between two integers, it 
must lie between two consecutive integers m - 1 and m for some m G Z 
where -m2 < m < mi. Thus m - 1 < nx < m implies that m < 1 + nx and nx < 
m. We therefore obtain 
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nx<m< l+nx<ny 
or, equivalently (since n 0), x < m/n < y. ■ 

Corollary Suppose x, y E R and assume that x < y. Then there exist integers 
m e Z and k > such that x < < y. 

Proof Simply note that the proof of Theorem 0.4 could be carried through if 
we choose an integer k > so that 2\y - x) > 1, and replace n by 2^ through- 
out. I 

In addition to the real number system IR we have been discussing, it is 
convenient to introduce the extended real number system as follows. To the 
real number system IR, we adjoin the symbols +oo and -oo which are defined to 
have the property that -oo < x < +oo for all x E IR. This is of great notational 
convenience. We stress however, that neither +oo or -oo are considered to be 
elements of R. 

Suppose A is a non-empty set of real numbers. We have already defined 
sup A in the case where A has an upper bound. If A is non-empty and has no 
upper bound, then we say that sup A = +oo, and if A = 0, then sup A = -oo. 
Similarly, if A ?£ and has no lower bound, then inf A = -oo, and if A = 0, 
then inf A = +oo. 

Suppose a, b G IR with a < b. Then the closed interval [a, b] from a to b is 
the subset of IR defined by 

[a,b] = {xe[R:a<x<b} . 

Similarly, the open interval (a, b) is defined to be the subset 

(a, b) = {x e R: a < X < b} . 

We may also define the open- closed and closed- open intervals in the obvious 
way. The infinity symbols ±oo thus allow us to talk about intervals of the 

form (-00, b], [a, +oo) and (-oo, +oo). 

Another property of the sup that will be needed later on is contained in the 
following theorem. By way of notation, we define R to be the set of all real 
numbers > 0, and R"^ = R"^ U {0} to be the set of all real numbers > 0. 

Theorem 0.5 Let A and B be non-empty bounded sets of real numbers, and 
define the sets 

A + B = {x + y:xEA and y E B} 
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and 

AB = {xy:xeAandyeB} . 

Then 

(a) For all A, B C IR we have sup (A + B) = sup A + sup B. 

(b) For all A, B C iR+ we have sup (AB) < (sup A)(sup B). 

Proof (a) Let a = sup A, (3 = sup B, and suppose x + y £ A + B. Then 

x + y < a + y < a + P 

so that a + |3 is an upper bound for A + B. Now note that given e > 0, there 
exists X £ A such that a - e/2 < x (or else a would not be the least upper 
bound). Similarly, there exists y £ B such that |3 - e/2 < y. Then a + |3 - e < 
X + y so that a + |3 must be the least upper bound for A + B. 

(b) If X G A C IR"^ we must have x < sup A, and if y G B C R"^ we have 
y < sup B. Hence xy < (sup A)(sup B) for all xy G AB, and therefore A 
and B 9i implies 

sup (AB) < (sup A)(sup B) . 
The reader should verify that strict equality holds if A C IR"^ and B C IR"^. I 

The last topic in our treatment of real numbers that we wish to discuss is 
the absolute value. Note that if x G R and x^ = a, then we also have (-x)^ = a. 

We define Va, for a > 0, to be the unique positive number x such that ^ = a, 
and we call x the square root of a. 

Suppose x, y > and let x^ = a and = b. Then x = Va, y = v^ and we 
have (Va Vb)^ = (xy)^ = x^ y^ = ab which implies that 

v^ = -v^Vb . 

For any a G R, we define its absolute value | a | by | a | = ^/sfi. It then fol- 
lows that I -a I = I a I, and hence 

, , r aif a>0 
' ' [-aifa<0 

This clearly implies that 

a < I a| . 

In addition, if a, b > and a < b, then we have (-Va)^ = a < b = (Vb)^ so that 
Va <Vb. 
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The absolute value has two other useful properties. First, we note that 
\ab\ = yj{abf = 4^ = 4c^ 4^ = \a\ \b\. 
Second, we see that 

|a + &|^ = [a + bY 

= a^+b^ + 2ab 

I |2 I |2 I I 

^ |a| +\b\ +2\ab\ 
= \a\ +\b\ +2\a\\b\ 
.{\aMb\f 

and therefore 

la + b| < lal + Ibl . 

Using these results, many other useful relationships may be obtained. For 
example, |a| = |a + b - b| < |a + b| + |-b| = |a + b| + |b| so that 

lal - Ibl < la + b| . 

Others are to be found in the exercises. 

Example 0.6 Let us show that if e > 0, then |x| < e if and only if -e < x < e. 
Indeed, we see that if x > 0, then |x| = x < e, and if x < 0, then |x| = -x < e 
which implies -e < x < (we again use the fact that a < b implies -b < -a). 
Combining these results shows that |x| < e implies -e < x < e. We leave it to 
the reader to reverse the argument and complete the proof. 

A particular case of this result that will be of use later on comes from let- 
ting X = a - b. We then see that |a - b| < e if and only if -e < a - b < e. 
Rearranging, this may be written in the form b-e<a<b + e. The reader 
should draw a picture of this relationship. / 

Exercises 

1. Prove that if A and B are countable sets, then A x B is countable. 

2. (a) A real number x is said to be algebraic (over the rationals) if it 
satisfies some polynomial equation of positive degree with rational 
coefficients: 
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x° + aji-i x° ^+---+aix + ao = 0. 

Given the fact (which we will prove in Chapter 6) that each polynomial 
equation has only finitely many roots, show that the set of all algebraic 
numbers is countable. 

(b) We say that a real number x is transcendental if it is not algebraic (the 
most common transcendental numbers are n and e). Using the fact that the 
reals are uncountable, show that the set of all transcendental numbers is 
also uncountable. 

3. If a, b > 0, show that Vab < (a + b)/2. 

4. For any a, b £ IR, show that: 

(a) ||a|-|b||<|a + b|. 

(b) I lal - Ibl I < la - b|. 

5. (a) If A C IR is nonempty and bounded below, show sup(-A) = -inf A. 
(b) If A C R is nonempty and bounded above, show inf(-A) = -sup A. 

0.5 INDUCTION 

Another important concept in the theory of sets is called "well-ordering." In 
particular, we say that a totally ordered set S is well-ordered if every non- 
empty subset A of S has a smallest element. For example, consider the set S of 
all rational numbers in the interval [0, 1]. It is clear that is the smallest 
element of S, but the subset of S consisting of all rational numbers > has no 
smallest element (this is a consequence of Theorem 0.4). 

For our purposes, it is an (apparently obvious) axiom that every non- 
empty set of natural numbers has a smallest element. In other words, the 
natural numbers are well-ordered. The usefulness of this axiom is that it 
allows us to prove an important property called induction. 

Theorem 0.6 Assume that for all n £ Z we are given an assertion A(n), and 
assume it can be shown that: 

(a) A(l) is true; 

(b) If A(n) is true, then A(n + 1) is true. 
Then A(n) is true for all n e Z . 

Proof If we let S be that subset of Z for which A(n) is not true, then we 
must show that S = 0. According to our well-ordering axiom, if S ^ then S 
contains a least element which we denote by N. By assumption (a), we must 
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have N 1 and hence N > 1. Since N is a least element, N - 1 ^ S so that 
A(N - 1) must be true. But then (b) implies that A(N) must be true which con- 
tradicts the definition of N. I 

Example 0.7 Let n > be an integer. We define n factorial, written n!, to be 
the number 

n! = n(n-l)(n-2)...(2)(l) 
with 0! defined to be 1. The binomial coefficient (g) is defined by 



k\{n-k)\ 



where n and k are nonnegative integers. We leave it to the reader (see 
Exercise 0.6.1) to show that 



In' 




V'' 


\n-k^ 



and 



k-\\ 



What we wish to prove is the binomial theorem 



*;=0 



We proceed by induction as follows. For n = 1, we have 



'1^ 



so that the assertion is true for n = 1. We now assume the theorem holds for n, 
and proceed to show that it also holds for n + 1. We have 



[x + yY ={x + y){x + y)^ ={x\y) 



k n-k 



/t=0 



k-¥\,n-k 



k=0 



k=0' 
k,n-k+l 



(*) 
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By relabelling the summation index, we see that for any function f with 
domain equal to {0, 1, . . . , n} we have 



n+l 



2 m = m + /(I) + • • • + m =2nk- 1). 



k=0 



We use this fact in the first sum in (*), and separate out the k = term in the 
second to obtain 

/ \n+l ^ \ k n-k+l , n+l , V k+l n-k+1 

We now separate out the k = n + 1 term from the first sum in this expression 
and group terms to find 



n+l , n+l , ,.n+l 
X + 



k n-k+l 

vc y 



kyK+l-k 



n+l I 



n + l 



^k^n+l-k 



as was to be shown. 



0.6 COMPLEX NUMBERS 

At this time we wish to formally define the complex number system C, 
although most readers should already be familiar with its basic properties. The 
motivation for the introduction of such numbers comes from the desire to 
solve equations such as + 1 = which leads to the square root of a negative 
number. We may proceed by manipulating square roots of negative numbers 
as if they were square roots of positive numbers. However, a consequence of 
this is that on the one hand, (a^^)^ = -1, while on the other hand 

= = v(-i)(-i) = v+r = 1 . 

In order to avoid paradoxical manipulations of this type, the symbol / was 

introduced by Euler (in 1779) with the defining property that z = -1. Then, if 
a > 0, we have = i Va. Using this notation, a complex number z G C is a 
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number of the form z = x+ iy where x £ IR is called the real part of z (written 
Re z), and y £ R is called the imaginary part of z (written Im z). 

Two complex numbers x + iy and u + iw are said to be equal if x = u and 
y = V. Algebraic operations in C are defined as follows: 

Addition: (x + iy) + (u + i\) = (x + u) + i{y + v). 
Subtraction: (x + iy) - (u + iw) = (x - u) + i{y - v). 
Multiplication: (x + /y)(u + iw) = (xu - yv) + /(xv + yu). 
Division: (x + /y)/(u + iw) = (x + /y)(u - zv)/(u + zv)(u - iw) 



It should be clear that the results for multiplication and division may be 
obtained by formally multiplying out the terms and using the fact that i^ = -1. 

The complex conjugate z* of a complex number z = x + jy is defined to 
be the complex number z* = x - iy. Note that if z, w £ C we have 



The absolute value (or modulus) |z| of a complex number z = x + jy is 
defined to be the real number 



By analogy to the similar result for real numbers, if z, w £ C then (using the 
fact that z = X + iy implies Re z = x < Vx^+y^ = |z| ) 



z + w\ ={z + w){z + w)* 

= ZZ* +ZW *+Z*W + WW 

= \z\ +2Re(zH'*) + |H'| 
^|z| +2|zh'*| + |h'| 
= |z| + 2|z||>v| + |>v| 
= i\z\ + \w\)' 



and hence taking the square root of both sides yields 



= [(xu + yv) + /(yu - vx)]/(u^ + v^). 



(z + w)* - z*+w * 
(zw)* = z*w* 
z + z* = 2Rez 
z-z* = 2/Imz 




2 



|z + w| < |z| + |w| 
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Let the sum zi + • • • + Zn be denoted by 2^=iZi. The following theorem is 
known as Schwartz's inequality. 

Theorem 0.7 Let ai , . . . , an and bi , . . . , bn be complex numbers. Then 







(ihf) 




7=1 









Proof Write (suppressing the limits on the sum) A = 2j lajP, B = 2j |bjP and 
C = 2j ajbj*. If B = 0, then bj = for all j = 1, . . . , n and there is nothing to 
prove, so we assume that B^O. We then have 

= 2i(Ba,-Cb,)iBa,*-Cbn 

= 2, \af-BC* a,b, * -BC 2, a, * b, + |C|' 2, f 
= B^A- B\cf - B\cf + \cf B 
= BiAB-\cf). 

But B > so that AB - |Cp > and hence ICp < AB. ■ 

It is worth our going through some additional elementary properties of 
complex numbers that will be needed on occasion throughout this text. Purely 
for the sake of logical consistency, let us first prove some basic trigonometric 
relationships. Our starting point will be the so-called "law of cosines" which 
states that c^ = a^ + b^ - 2ab cos 6 (see the figure below). 




b 



A special case of this occurs when 9 = Jt/2, in which case we obtain the 
famous Pythagorean theorem a^ + b^ = c^. (While most readers should already 
be familiar with these results, we prove them in Section 2.4.) 

Now consider a triangle inscribed in a unit circle as shown below: 
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y 





/ l\ 


a \ 


X 











The point P has coordinates (xp , yp) = (cos a, sin a), and Q has coordinates 
(xq, yq) = (cos (3, sin (3). Applying the Pythagorean theorem to the right tri- 
angle with hypotenuse defined by the points P and Q (and noting xq^ + yg^ = 
xp^ + yp^ = 1), we see that the square of the distance between the points P and 
Q is given by 

mf=i.XQ-Xpf+{yQ-ypf 

= i^Q +yQ) + i^p^ +yp^)- 2(XpXQ + ypyg ) 
= 2 - 2(cos a cos )8 + sin a sin 

On the other hand, we can apply the law of cosines to obtain the distance PQ, 
in which case we find that (PQ)^ = 2 - 2cos(a - p). Equating these expres- 
sions yields the basic result 

cos(a - |3) = cos a cos P + sin a sin p . 

Replacing p by -P we obtain 

cos(a + P) = cos a cos p - sin a sin p . 

If we let a = Jt/2, then we have cos(jt/2 - p) = sin p, and if we now replace 
P by Jt/2 - p, we find that cos P = sin(jt/2 - P). Finally, we can use these last 
results to obtain formulas for sin(a + P). In particular, we replace P by a + P 
to obtain 
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sin(a + P) = cos(;r l2-{a + P)) 
= cos(;r 12-a-P) 

= cos(;r/2-a)cos)3 + sin(;r/2-«)sm/3 
= sinacos)8 + cosasin)8 

Again, replacing p by -p yields 

sin(a - P) = sin a cos P - cos a sin p . 

(The reader may already know that these results are simple to derive using the 
Euler formula exp(+i0) = cos + isin which follows from the Taylor series 
expansions of exp x, sin x and cos x, along with the definition i - - \.) 

It is often of great use to think of a complex number z = x + jy as a point 
in the xy-plane. If we define 

r = |z| = Vx2 + y2 

and 

tan = y/x 

then a complex number may also be written in the form 

z = X + jy = r(cos + /sin 0) = r exp(j0) 
(see the figure below). 







z = X + jy 






y 




X 



Given two complex numbers 

Zi = ri(cos 01+ /sin 0i) 

and 

Z2 = r2(cos 02 + Jsin 02) 
we can use the trigonometric addition formulas derived above to show that 
Z1Z2 = ri r2[cos(0i + 02) + isin(0i + 02)] . 
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In fact, by induction it should be clear that this can be generalized to (see 
Exercise 0.6.5) 

Zj Z2 • • • Zn = 

Ti r2 • • • rn[cos(0i + 02 + • • • + 9n) + «sin(6i + 62 + • • • + 6n)] • 
In the particular case where Zi = • • • = Zn, we find that 

z" = r"(cos n0 + /sin n0) . 

This is often called De Moivre's theorem. 

One of the main uses of this theorem is as follows. Let w be a complex 
number, and let z = w° (where n is a positive integer). We say that w is an nth 

root of z, and we write this as w = z^^°. Next, we observe from De Moivre's 
theorem that writing z = r(cos + /sin 0) and w = s(cos (|) + /sin (|)) yields 
(assuming that z 0) 

r(cos + /sin 0) = s"(cos ncj) + /sin ncj)) . 

But cos = cos(0 ± 2kjt) for k = 0, ±1, ±2, . . . , and therefore r = s" and n(j) = 
± 2kjr. (This follows from the fact that if Zi = Xi + iyi = ri(cos 0i + /sin 0i) 
and Z2 = X2 + /y2 = r2(cos 02 + /sin 02), then Zi = Z2 implies Xi = X2 and yi = y2 
so that ri = r2, and hence 0i = 02.) Then s is the real positive nth root of r, and 
(j) = 0/n ± 2kjt/n. Since this expression for (|) is the same if any two integers k 
differ by a multiple of n, we see that there are precisely n distinct solutions of 

z = w" (when z ^ 0), and these are given by 

w = ri^"[cos(0 + 2kjt)/n + /sin(0 + 2kjt)/n] 
where k = 0, 1, . . . , n - 1. 

Exercises 
1. (a) Show 











- 





(b) Show 



















^k-\] 
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2. Prove by induction the formula 1 + 2 + • • • + n = n(n + l)/2. 

3. Prove by induction the formula 

\ + X + X +--- + X = 

1-x 

where x is any real number ^ 1. 

4. Prove by induction that for any complex numbers Zi, . . . , Zn we have: 
(a) 

n 

(b) |ZiZ2 • • • Znl = IZillZjl • • • |Zn|. 

5. Prove by induction that for any complex numbers Zi, . . . , Zn we have 

Z1Z2 • • • Zn = ViVj • • • rn[C0S(6i + 62 + • • • + 9n) 

+ / sin(6i + 02 + • • • + 6n)] 

where zj = rj exp(i6j). 

0.7 ADDITIONAL PROPERTIES OF THE INTEGERS 

The material of this section will be generalized in Sections 6.1 and 6.2 to the 
theory of polynomials. However, it will also be directly useful to us in our 
discussion of finite fields in Section 6.6. Most of this section should be famil- 
iar to the reader from very elementary courses. 

Our first topic is the division of an arbitrary integer a £ Z by a positive 

integer b £ Z For example, we can divide 11 by 4 to obtain 11= 2-4 + 3. As 
another example, -7 divided by 2 yields -7 = -4-2 + 1. Note that each of these 
examples may be written in the form a = qb + r where q G Z and < r < b. 
The number q is called the quotient, and the number r is called the 
remainder in the division of a by b. In the particular case that r = 0, we say 
that b divides a and we write this as b|a. If r 0, then b does not divide a, and 
this is written as b|a. If an integer p G Z"^ is not divisible by any positive 
integer other than 1 and p itself, then p is said to be prime. 
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It is probably worth pointing out the elementary fact that if a|b and a|c, 
then a|(mb + nc) for any m, n £ Z. This is because a|b implies b = qia, and a|c 
implies c = qja. Thus mb + nc = (mqi + nq2)a so that a|(mb + nc). 

Theorem 0.8 (Division Algorithm) If a G Z and b E Z then there exist 
unique integers q and r such that a = qb + r where < r < b. 

Proof Define S = {a - nb > 0: n G Z}. In other words, S consists of all non- 
negative integers of the form a - bn. It is easy to see that S ^0. Indeed, if a > 
we simply choose n = so that a G S, and if a < we choose n = a so that 
a - ba = a(l - b) G S (since a < and 1 - b < 0). Since S is a nonempty subset 
of the natural numbers, we may apply the well-ordering property of the 
natural numbers to conclude that S contains a least element r > 0. If we let q 
be the value of n corresponding to this r, then we have a - qb = r or a = qb + r 
where < r. We must show that r < b. To see this, suppose that r > b. Then 

a-(q+l)b = a-qb-b = r-b > 

so that a - (q + l)b G S. But b > so that 

a - (q + l)b = (a - qb) - b<a-qb = r 

which contradicts the definition of r as the least element of S. Hence r < b. 

To prove uniqueness, we suppose that we may write a = qib + rj and a = 
qjb + V2 where < rj < b and < rj < b. Equating these two formulas yields 
qib + ri = qjb + rj or (q, - q2)b = rj - ri, and therefore b|(r2 - ri). Using the 
fact that < r, < b and < rj < b, we see that rj - r, < b - r, < b. Similarly we 
have ri - r2 < b - r2 < b or -b < r2 - x^. This means that -b < r2 - r, < b. 
Therefore r2 - ri is a multiple of b that lies strictly between -b and b, and thus 
we must have rz - ri = 0. Then (qi - q2)b = with b^Q, and hence qi - q2 = 
also. This shows that ri = rj and qi = q2 which completes the proof of 
uniqueness. I 

Suppose we are given two integers a, b G Z where we assume that a and b 
are not both zero. We say that an integer d G Z is the greatest common divi- 
sor of a and b if d|a and d|b, and if c is any other integer with the property that 
c|a and c|b, then c|d. We denote the greatest common divisor of a and b by 
gcd{a, b}. Our next theorem shows that the gcd always exists and is unique. 
Furthermore, the method of proof shows us how to actually compute the gcd. 
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Theorem 0.9 (Euclidean Algorithm) If a, b £ Z are not both zero, then 

there exists a unique positive integer d £ Z such that 

(a) d|a and d|b. 

(b) If c E Z is such that c|a and c|b, then c|d. 

Proof First assume b > 0. Applying the division algorithm, there exist unique 
integers qi and ri such that 

a = qib + ri with < ri < b . 

If ri = 0, then b|a and we may take d = b to satisfy both parts of the theorem. If 
ri ^ 0, then we apply the division algorithm again to b and ri, obtaining 

b = qjri + rj with < rj < ri . 

Continuing this procedure, we obtain a sequence of nonzero remainders ri, rj, 
. . . , rk where 



a 




with 


£ ri < 


b 


= ^21 + h 


with 


£ < 


h 


= q^r^ + 


with 


^ ^3 < 


-2 




with 




:-l 









(*) 



That this process must terminate with a zero remainder as shown is due to the 
fact that each remainder is a nonnegative integer with ri > r2 > • • • . We have 
denoted the last nonzero remainder by r^. 

We now claim that d = r^. Since r^-i = qk+irk, we have rklr^-] . Then, 
because rk-2 = qkfk-i + and rk|rk and rklr^-i , we have rk|rk-2 . Continuing 
in this manner, we see that rk|rk_i, rk|rk_2 , . . . , rk|ri, rk|b and rk|a. This shows 
that rk is a common divisor of a and b. To show that rk is in fact the greatest 
common divisor, we first note that if c|a and c|b, then c|ri because ri = a - qib. 
But now we see in the same way that c|r2, and working our way through the 
above set of equations we eventually arrive at c|rk. Thus rk is a gcd as claimed. 

If b < 0, we repeat the above process with a and -b rather than a and b. 
Since b and -b have the same divisors, it follows that a gcd of {a, -b} will be 
a gcd of {a, b} (note we have not yet shown the uniqueness of the gcd). If b = 
0, then we can simply let d = |a| to satisfy both statements in the theorem. 

As to uniqueness of the gcd, suppose we have integers di and d2 that sat- 
isfy both statements of the theorem. Then applying part (b) to both di and d2. 
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we must have di|d2 and d2|di. But both di and d2 are positive, and hence di = 
d2. I 

Corollary If d = gcd{a, b} where a and b are not both zero, then d = am + bn 
for some m, n G Z. 

Proof Referring to equations (*) in the proof of Theorem 0.9, we note that 
the equation for r^- 2 may be solved for rk to obtain rt = r^- 2 - rjc-i Qk- Next, 
the equation r^-s = qk-irk-2 + ^k-i may be solved for r^-i, and this is then 
substituted into the previous equation to obtain r^ = rk-2(l + qk-iqk) - rk-3 qk- 
Working our way up equations (*), we next eliminate rk-2 to obtain r^ in 
terms of rk-3 and rk- 4 . Continuing in this manner, we eventually obtain r^ in 
terms of b and a. I 

If a, b G Z and gcd{a, b} = 1, then we say that a and b are relatively 
prime (or sometimes coprime). The last result on integers that we wish to 
prove is the result that if p is prime and p|ab (where a, b G Z), then either p|a 
or p|b. 

Theorem 0.10 (a) Suppose a, b, c G Z where a|bc and a and b are relatively 
prime. Then a|c. 

(b) If p is prime and ai, . . . , an G Z with p|ai • • • an, then p|ai for some i = 
1, . . . ,n. 

Proof (a) By the corollary to Theorem 0.9 we have gcd{a, b} = 1 = am + bn 
for some m, n G Z. Multiplying this equation by c we obtain c = amc + bnc. 
But a|bc by hypothesis so clearly a|bnc. Since it is also obvious that a|amc, we 
see that a|c. 

(b) We proceed by induction on n, the case n = 1 being trivial. We there- 
fore assume that n > 1 and p|ai • • • an. If p|a, • • • an-i, then p|ai for some i = 1, 
. . . , n - 1 by our induction hypothesis. On the other hand, if p|ai • • • an-i 
then gcd{p, ai, . . . , an-i} = 1 since p is prime. We then apply part (a) with a = 
p, b = ai • • • an-i and c = an to conclude that p|an. ■ 

Exercises 

1. Find the gcd of the following sets of integers: 

(a) {6,14}. 

(b) {-75, 105}. 

(c) {14,21,35}. 
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2. Find the gcd of each set and write it in terms of the given integers: 

(a) {1001,33}. 

(b) {-90, 1386}. 

(c) {-2860,-2310}. 

3. Suppose p is prime and p|a where a £ Z. Prove that a and p are relatively 
prime. 

4. Prove that if gcd{a, b} = 1 and c|a, then gcd{b, c} = 1. 

5. If a, b G Z+, then m G Z + is called the least common multiple (abbre- 
viated 1cm) if a|m and b|m, and if c G Z is such that a|c and b|c, then m|c. 
Suppose a = pi^' • • • Pk^"^ and b = p/' • • • Pk'"*^ where pi, . . . , pk are distinct 
primes and each Si and ti are > 0. 

(a) Prove that a|b if and only if Sj < tj for all i = 1 , . . . , k. 

(b) For each i = 1, . . . , k let Uj = min{Si, ti} and Vj = max{Si, ti}. Prove 

that gcd{a, b} = pi'^^ • • • Pk*"" and lcm{a, b} = pi^i • • • Pk""^ . 

6. Prove the Fundamental Theorem of Arithmetic: Every integer > 1 can 
be written as a unique (except for order) product of primes. Here is an out- 
line of the proof: 

(a) Let S = {aGZ:a>l and a can not be written as a product of primes.} 
(In particular, note that S contains no primes.) Show that S = by 
assuming the contrary and using the well-ordered property of the natural 
numbers. 

(b) To prove uniqueness, assume that n > 1 is an integer that has two dif- 
ferent expansions as n = pi • • • ps = qi • • • qt where all the pi and q, are 
prime. Show that pilqj for some j = 1, . . . , t and hence that pi = qj. Thus 
pi and qj can be canceled from each side of the equation. Continue this 
argument to cancel one pi with one qj, and then finally concluding that s = 
t. 



CHAPTER 1 



An Introduction to Groups 



While we have no intention of presenting a comprehensive treatment of group 
theory in this text, there are a number of definitions that will facilitate a rig- 
orous description of vector spaces. Furthermore, the concepts from abstract 
algebra that we shall introduce will be of great use to us throughout the text. 

1.1 DEFINITIONS 

A group (G, •) is a nonempty set G together with a binary operation called 
multiplication (or a product) and denoted by • that obeys the following 
axioms: 

(Gl) a, b G G implies a»b G G (closure); 
(G2) a, b, c G G implies (a»b)»c = a«(b«c) (associativity); 
(G3) There exists e G G such that a»e = e»a = a for all a G G (identity); 
(G4) For each a G G, there exists a"' G G such that a»a"' = a"' "a = e 
(inverse). 

Furthermore, a group is said to be abelian if it also has the property that 
(G5) a«b = b«a for all a, b G G (commutativity). 
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In the case of abelian groups, the group multiplication operation is fre- 
quently denoted by + and called addition. We will generally simplify our 
notation by leaving out the group multiplication symbol and assuming that it 
is understood for the particular group under discussion. 

The number of elements in a group G is called its order of and will be 
denoted by o(G). (The order of G is frequently denoted by |G| although we 
shall not use this notation.) If this number is finite, then we say that G is a 
finite group. Otherwise, G is said to be infinite. 

While we have defined a group in the usual manner, it should be realized 
that there is a certain amount of redundancy in our definition. In particular, it 
is not necessary to require that a "right inverse" also be the "left inverse." To 
see this, suppose that for any a £ G, we have the right inverse defined by 
aa'' = e. Then multiplying from the left by a"' yields a^'aa"' = a"'. But a"' G G 
so there exists an (a"')"' G G such that (a"')(a"')"' = e. Multiplying our 
previous expression from the right by (a"')"' results in a"' a = e, and hence we 
see that a"^ is also a left inverse. Of course, we could have started with a left 
inverse and shown that it is also a right inverse. 

Similarly, we could have defined a right identity by ae = a for all a G G. 
We then observe that a = ae = a(a"'a) = (aa"')a = ea, and hence e is also a left 
identity. 

It is easy to show that the identity element is unique. To see this, suppose 
that there exist e, e G G such that for every a G G we have ea = ae = ea = ae = 
a. Since ea = a for every a G G, we have in particular that ee = e. On the other 
hand, since we also have ae = a, it follows that ee = e. Therefore e = ee = e so 
that e = e. 

Before showing the uniqueness of the inverse, we first prove an important 
basic result. Suppose that ax = ay for a, x, y G G. Let a"' be a (not necessarily 
unique) inverse to a. Then x = ex = (a"'a)x = a"' (ax) = a"' (ay) = (a"'a)y = ey = 
y. In other words, the equation ax = ay means that x = y. This is sometimes 
called the (left) cancellation law. As a special case, we see that aa"' = e = aa"' 
implies a"' = a'' so that the inverse is indeed unique as claimed. This also 
shows that 

(a-')-' = a 

since (a"') "'(a"') = e and aa"' = e. 

Finally, another important result follows by noting that (ab)(b"'a"') = 
a((bb"')a"') = a(ea"') = aa"' = e. Since the inverse is unique, we then see that 

(ab)-' = b"'a"' . 

This clearly extends by induction to any finite product of group elements. 
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Example 1.1 The set of integers Z = 0, +1, +2, . . . forms an infinite abelian 
group where the group multiplication operation is just ordinary addition. It 
should be obvious that the (additive) identity element is 0, and the inverse of 
any number n is given by -n. However, it is easy to see that Z is not a group 
under the operation of ordinary multiplication. Indeed, while Z is both closed 
and associative under multiplication, and it also contains the (multiplicative) 
identity element 1, no element of Z (other than +1) has a multiplicative 
inverse in Z (for example, 2"' = 1/2 ^ Z). 

On the other hand, if we consider the set Q of all rational numbers, then Q 
forms a group under ordinary addition (with identity element and inverse 
-p/q G Q to any p/q G Q). Moreover, the nonzero elements of Q also form a 
group under ordinary multiplication (with identity element 1 and inverse 
q/p e Q to any p/q e Q). / 

Example 1.2 A more complicated (but quite useful) example is given by the 
set of all rotations in the xy-plane. (This example uses some notation that we 
have not yet defined in this book, although most readers should have no diffi- 
culty following the discussion.) Consider the following figure that shows a 
vector f = (x, y) making an angle ^ with the x-axis, and a vector f ' = (x', y') 
making an angle 6 + (|) with the x-axis: 



We assume r = | f | = | f '| so that the vector f ' results from a counterclockwise 
rotation by an angle 6 with respect to the vector f . From the figure, we see 
that f has components x' and y' given by 




f" = (x', y') 



f = (x, y) 



x' X 



jc ' = r cos(0 + 0) = r cos cos - r sin sin = X cos d-ysind 
= rsin(0 + 0) = rsin0cos(^ + rcos0sin0 = xsinS + ycosS. 



Let R(a) denote a counterclockwise rotation by an angle a. It should be 
clear that R(0) is just the identity rotation (i.e., no rotation at all), and that the 
inverse is given by R(a)"' = R(-a). With these definitions, it is easy to see 
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that the set of all rotations in the plane forms an infinite (actually, continuous) 
abelian group. A convenient way of describing these rotations is with the 



matrix 



R(a) = 



(cos a -sina^ 
I sina cosce i 



(Such a matrix is said form a representation of the rotation group.) We then 
see that r' = R(6) f , which in matrix notation is just 



''^'^ ''cos© -sind^^ 
sind cos 6 



X 



,1 



Using this notation, it is easy to see that R(0) is the identity since 



,1 



and also that R(e)-' = R(-0) because 



Rid)R{-d) = 



(cos 6 -sia6\ 
sind cos 6 



cosd sind) 
-sind cosd I 



(I 0' 
I, 



\ = R{-d)R(d). 



We remark that while the rotation group in two dimensions is abelian, the 
rotation group in three dimensions is not. For example, let Rz(6) denote a rota- 
tion about the z-axis (in the "right-handed sense"). Then, applied to any vector 
X lying along the x-axis, we see that 



Ry(90°)Rz(45°)x ^ Rz(45°)Ry(90°)x 

since in the second case, the result lies along the z-axis, while in the first case 
it does not. / 



While we will return shortly to discuss subgroups in more detail, it will be 
of use to define them now. If G is a group, then a subset H C G is said to be a 
subgroup of G if the elements of H form a group under the same group multi- 
plication rule as G. For example, the set Z of integers is a subgroup of the 
group Q of all rational numbers under ordinary addition. Furthermore, it is 
easy to show that a nonempty subset H of a group G is a subgroup of G if and 
only if a, b E H implies that ab G H, and a G H implies that a"' G H (see 
Exercise 1.1.9). 
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Exercises 



1. Decide which of the following sets G forms a group under the indicated 
operation. If G does not form a group, give the reason. 

(a) G = {all integers} under ordinary subtraction. 

(b) G = {all nonzero rational numbers} under ordinary division. 

(c) G = {ao, ai, . . . , ag} where 



(d) G = {2™ 3": m, n £ Z} under ordinary multiplication. 

2. Let F denote the set of all mappings from IR into R. For any f, g G F we 
define (f + g)(x) = f(x) + g(x) for each x G IR so that f + g G F. Show that 
this defines a group. 

3. Show that the collection of all subsets of a set S, with the operation of 
taking symmetric differences (see Exercise 0.1.2) as the group multipli- 
cation operation, forms a group. [Hint: Show that the identity element is 
0, and the inverse of any A C S is A itself.] 

4. Prove that any group of order n < 4 must be abelian. 

5. Given two groups A and B, we can form the Cartesian product A x B = 
{(a, b): a G A and b G B} of these groups considered as sets. Prove that 
A X B can be made into a group with respect to the operation defined by 
(ai, bi)(a2, bj) = (aiaj, bib2) for all ai, aj G A and bi, G B. This group is 
called the direct product of A and B. 

6. Prove that {(x, x): x G G} is a subgroup of G x G (see the previous 
problem). This is called the diagonal subgroup of G x G. 

7. Let G = {gi, . . . , gn} be a group, and let h G G be arbitrary but fixed. 
Define the set hG = {hgi, . . . , hgn} = {ghp • • • , gh„}. Show that hG = G, 
and conclude that the ordered set (hi, ... , hn) is a permutation of the 
ordered set (1, . . . , n). (This simple but very useful result is frequently 
referred to as the rearrangement lemma.) 

8. Let H be a subgroup of a group G. 
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(a) If e is the identity element in G and f is the identity element in H, 
show that f = e. 

(b) If a £ H, show that the inverse element a"' is the same in H as the a"' 
is in G. 

9. Let H be a nonempty subset of a group G. Prove that H is a subgroup of 
G if and only if a, b £ H implies ab £ H and a £ H implies a"' £ H. 

10. Let iHhe a collection of subgroups of a group G. Show that the intersec- 
tion of all H E is a subgroup of G. 

11. Let G be a group. An element a E G is said to be conjugate to an element 
b E G if there exists g E G such that b = gag"'. Show that this defines an 
equivalence relation on G. (Mutually conjugate elements of G are said to 
form a (conjugate) class.) 

12. Let X be a (nonempty) subset of a group G, and let {Hj: i e 1} be the 
collection of all subgroups of G that contain X. Then flHi is called the 
subgroup of G generated by the set X and denoted {X>. Prove that {X> 
consists of all finite products a^^^a^^ • ■ ■ aj^'' where a,- G X and niGT. 
[Hint : Show that the set H of all such products is a subgroup of G that 
contains X and is contained in every subgroup containing X. Thus H < 
(XX H.] 

1.2 PERMUTATION GROUPS 

Let G be any group and suppose a E G. As a matter of notational conven- 
ience, we define a^ = e, a^ = a, a^ = aa, . . . , a'' = aa^~^, as well as a~^ = (a"')^, 
a~^ = (a"')^, . . . (where a"' is the usual inverse element to a). It is then easy to 
see that for any m, n G Z we have a*" a" = a™"^" and (a™)" = a™". From now 
on we will assume the reader understands that this is what is meant when we 
write an element of any group to a power. 

Now consider three objects (, A, O) where the parentheses mean that the 
given order is relevant. We define this to be the canonical (or standard) order 
on the set {, A, O}. Given any other ordered triple, for example (O, , A), we 
define a permutation f of the set S = {, A, 0} by 
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where the first line is the set of objects in their canonical order and the second 
line is the given order. In other words, a permutation on a set S is a bijection 
from S onto itself. Note that simply giving an arbitrary order to a collection of 
objects does not in itself define a permutation. It is necessary that some canon- 
ical order also be specified as a point of reference. 

This notation, where the top row defines the canonical order, is referred to 
as two-line notation. However, it is very important to realize that as long as 
the same pairing of objects is maintained between the top and bottom rows, 
we may rearrange these pairs any way we please. For example, the above 
permutation f can equally well be written as 

(0 □ A\ 

/ = 

A O □ 



It is also common to use a simplified one-line notation. In this case, the 
canonical order must be understood. For example, in the first case above we 
would write simply f = (O, , A) where the canonical order is understood to be 
(, A,0). 

While we have now given a precise definition of the term "permutation," 
there are other ways of describing permutations that are very useful in prac- 
tice. Two of these are given in the next (rather long) example, which will then 
be generalized to form one of the most useful groups in linear algebra. 

Example 1.3 Suppose we have three boxes that each contain a single object. 
Now, given three distinct objects and three boxes, any one of the three objects 
could go into the first box, then either of the two remaining objects could go 
into the second box, and finally only the remaining object can go into the third 
and last box. In other words, there are 3 ! = 6 possible placements of the three 
distinct objects in the three boxes such that each box receives a single object. 
Let us see how permutations can be used to describe the distribution of 
distinct objects among boxes. We give two common, intuitive interpretations. 

Imagine three boxes labelled 1, 2, 3 that contain objects xi, X2, X3 respec- 
tively, as shown below: 



X 



1 2 3 

We now redistribute these objects among the boxes as follows: 



Y 
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One way to describe the transition from the first distribution to the second is 
by the permutation 

'1 2 3\ 



/ = 



2 3 1 



which is to be interpreted as a rule for redistributing objects by saying "take 
the object in box i (a number in the upper row) and place it in box f (i) (the 
number in the lower row directly below it)." In this example, this means that 
we take the object in box i = 1 and place it in box f (1) = 2, the object in box 
i = 2 goes into box f (2) = 3, and the object in box i = 3 goes into box f (3) = 
1. This rule yields the second distribution from the first. 

(Note also that in terms of our original definition of a permutation, we can 
interpret f as a reordering of boxes in space. In other words, we can equally 
well describe the above redistribution in effect by leaving the objects fixed in 
space and rearranging the boxes underneath them. It is easy to see that if we 
leave the objects in the order (xi, X2, X3) and label the boxes underneath them 
in the order (2, 3, 1), then we obtain the same pairing of objects and boxes.) 

Another approach to describing this transition is by using permutations on 
the set of objects. For example, if we let 



/ = 



^ "V Y V ^ 

1 2 3 
\-^3 -^1 -^2 / 



then the second distribution (the lower row) is obtained from the first distribu- 
tion (the upper row) by interpreting f as "replace object xi (wherever it is) by 
object X3 , replace object X2 (wherever it is) by object xi , and replace object 
X3 (wherever it is) by object X2." An equivalent way to describe this 
permutation is by the mapping f defined by 

/(^l) = ^3 

7(^2) = ^1 

fix J ) = ^2 

which we can write in the simple one-line notation 

f ~ (-^3' -^1' ^2)' 

Let us denote the set of objects {xi, X2, X3} by S. Since there are only six 
possible distinct arrangements of S within the three boxes, there can be only 
six such permutations of S. We wish to make this set of permutations into a 
group. In particular, we will then have a group (denoted by S3) of permuta- 
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tions defined on the set S. This group is called the symmetric group (or the 
permutation group) of degree 3. Since S3 contains 3! = 6 elements, its order 
is 6. 

We define the group multiplication as the composition of our permuta- 
tions. For example, consider the permutation in S3 defined by either 



8 = 



(I 2 3\ 
2 1 3 



or 



^ V V "V ^ 

1 2 3 
\-^2 -^1 -^3 / 



which in our one-Une notation is simply g = (x2, xi, X3). Composing this with 
the above permutation f = (X3, xi, X2) we have, for example, 

(fg)(Xi) = f(g(Xi)) = f(X2) = XI 

and it is easy to see that the complete expression is given by fg = (xi, X3, X2). 
Note however, that 

gf = (X3,X2,Xi) 9i fg 

so that S3 is a nonabelian group. This composition of mappings also shows us 
how to multiply our permutations. Indeed, if we write out the equation fg = 
(xi, X3, X2) in terms of our two-line notation, we obtain 




Reading the product from right to left, we first see that xi is replaced by X2, 
and then this X2 is replaced by xi, and the net result is that xi is replaced by xi. 
Next we see that X2 is first replaced by X] , and then this X] is replaced by X3 
with the net result of replacing X2 by X3 . Finally, X3 is replaced by X3 , and 
then this X3 is replaced by X2, resulting in the replacement of X3 by X2. 
Therefore we see that combining the product from right to left results in 
exactly the same permutation as shown on the right hand side. 

Now let us see how to combine the alternative descriptions in terms of f 
and g . We know that f takes the initial distribution 



T 



X2 

~2 



■^3 

3 
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to the redistributed form ("contents of box 1 to box 2, contents of box 2 to box 
3, and contents of box 3 to box 1") 



1 



1 2 3 

and g takes the initial distribution to the redistributed form 



X 



1 



1 2 3 

Applying f to this last distribution we obtain (just take the contents of box 1 
to box 2 etc.) 



X 



1 



1 2 3 

With respect to the initial distribution, this composition of permutations is just 
the permutation 

(I 2 3\ 
3 2 1, 



In other words, simply following each permutation in sequence results in 



f8 = 



1 2 

2 3 



2 3\ 



1 3 




Again reading the product from right to left, we see that the object in box 1 
goes into box 2, and then the object in box 2 goes into box 3, with the net 
result that the object in box 1 goes into box 3. Next, the object in box 2 goes 
into box 1, and then the object in box 1 goes into box 2, resulting in the object 
in box 2 going into box 2. Finally, the object in box 3 goes into box 3, and 
then the object in box 3 goes into box 1, resulting in the object in box 3 going 
into box 1. Therefore, reading this type of product from right to left also 
results in the correct combination of permutations. 

We now observe that f ^(xi) = f(x3) = X2, and in general 



f^ = (X2, X3,Xi) 



and 



f-^ = (Xi, X2, X3) 
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which shows that f ^ = ff ^ = e, and hence f ' = f ^. Similarly, we leave it to the 
reader to show that = e, and hence g"' = g. 

Since S3 contains six elements and we have already constructed the six 

distinct mappings {e, f, g, f ^, fg, gf}, it must be true that any combination of 
mappings may be reduced to one of these six. To see this, all we really need to 
calculate is (f"'g)(xi) = f"'(x2) = X3, and in general, 

f 'g = (X3, X2, xi) = gf 

so that f"^g = gf. For example, we have f(gf) = f(f"'g) = (ff~Og = g. Other 
combinations are proved in a similar manner. / 

We now generalize this example to the case of an arbitrary (but finite) 
number of elements. Let S be a set containing a finite number n of elements. 
Then the set Sn of all one-to-one mappings of S onto itself is called the 
permutation group of degree n. It should be clear that Sn is of order n!. If 
f £ Sn, then f has the effect of taking Xj f(Xi) which we may write as 

^ _ ^1 ^2 "' 

where (ii, . . . , in) is some permutation of (1, . . . , n). To simplify our nota- 
tion, let us write this mapping as 

fl 2 ■■■ n\ 
[h h ••• in) 

where the top row stands for (xi, X2, . . . , Xn) and the bottom row represents 
(xip Xij, . . . , Xi^) which is just (xi, . . . , Xn) in some permuted order. This 
should not be confused with the interpretation (which we will no longer use) 
of permutations as "the object in box 1 goes into box ii" etc. 
The identity element in Sn is 

fl 2 ■■■ n\ 
[l 2 ••• nj 

and the inverse to any given permutation is just the permutation that restores 
the original order. For instance, the inverse to the permutation f defined in 
Example 1.3 is the permutation f"' = f ^ given in this notation by 
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In general, we will denote elements of Sn by Greek letters such as 6, (|) and 
o, so that we have expressions such as 6x, 6\ = 66x, and so forth. In other 
words, if 6 G S3 is just the mapping f in the previous example, then we would 
have 61 = 3, 62 = 1 and 63 = 2. 

Now let S be any set of n elements, and consider any element 6 G Sn. 

Given any x, y G S, we say that x is equivalent to y if y = 6' x for some i G Z, 
and we write this as x =g y. Since x = 6 ^ x = ex = x, we see that x x. Next, 
note that if x ==0 then x = 6^ y so that y = 6~^x, and hence y ~e x. In addition, 

if X =0 y and y ~q z, then x = 6^ y = 6' 6-'z = 6'"^-' z, and hence x =9 z. We have 
therefore defined an equivalence relation on S as described in Section 0.3. 
Furthermore, Theorem 0.2 shows that this equivalence relation induces a 
decomposition of S into disjoint subsets called the equivalence classes of S. 

For each x G S, the equivalence class of x is the set [x] = {6' x: i G Z} 
which is called the orbit of x under 6. Since S is finite, sooner or later 
repeated applications of 6 to x must give back x. In other words, for each x G 

S there exists some smallest positive integer m such that 6™x = x (where the 
value of m need not be the same for every x G S). Thus the orbit of x under 6 
will be the set {x, 6x, . . . , 6™-ix}. If we consider these elements as being in a 
particular order, we then obtain what is called a cycle of 9, and we write this 
as (x, 6x, . . . , O'^'^x). In words, this means "x is replaced by 6x, 6x is 
replaced by 6^x, . . . , and O'^'^x is replaced by x." It should be clear that a 
knowledge of all the cycles of 6 is the same as knowing 6, because we would 
then know the result of applying 6 to any x G S. (While the cycle notation is 
the same as the one-line notation for a permutation, the context should always 
make it clear which is meant.) 

Example 1.4 Let S = {xi, . . . , xe} which we denote by {1, . . . , 6} for 
simplicity. We consider the element 6 G Se given by 




1 2 3 4 5 6\ 

2 1 3 5 6 4]' 



Now observe that 61 = 2 and 6^1 = 62 = 1, so the orbit of 1 is the set {1, 2} 
and the corresponding cycle is (1, 2). Since this cycle is the equivalence class 
of 1 and equivalence classes are disjoint, we see that it must also be the 
equivalence class of 2. Continuing, the orbit of 3 is just {3}, and for 4 we have 
64 = 5, 6^4 = 6, and 6^4 = 4 so that the orbit of 4 is {4, 5, 6}. Thus the cycles 
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of 6 are (1, 2), (3) and (4, 5, 6). Notice that these cycles are disjoint ordered 
equivalence classes of S under the mapping 6 £ Se. / 

We can carry this idea one step further as follows. Consider a cycle of the 
form (ii, . . . , im) which we now interpret as that permutation which replaces 
ii by i2 , i2 by i 3 , . . . , im-i by im, and im by ii. For example, using the set 
S = {1, . . . , 6}, the cycle (2, 6, 3) is to be interpreted as the permutation 

1 2 3 4 5 6\ 
1 6 2 4 5 3]' 



Since we already know how to multiply permutations, we now have a way to 
multiply cycles. Thus, using this same S and, for example, the cycles (1, 5) 
and (2, 6, 3), we have 

(1,5)(2, 6, 3) = 




Note that while we have defined our multiplication as proceeding from right 
to left, in this case we would have obtained the same result by multiplying the 
cycles in either order. In fact, it should not be hard to convince yourself that 
this will always be the case when disjoint cycles are multiplied together. In 
other words, disjoint cycles commute. This is because each cycle only acts on 
a specific subset of elements that are not acted on by any other (disjoint) 
cycle. 

As another example, let us now find the cycles of the permutation 



e = 



'1 2 3 4 5 6^ 
^5 6 2 4 1 3] 



We have 61 = 5 and 6^1 = 1 so that the orbit of 1 is {1, 5}. Also, 62 = 6, 6^2 = 
3, and 6 2 = 2 so the orbit of 2 is {2, 6, 3}. Therefore 6 has the cycles (1,5) 
and (2, 6, 3) (and of course, also (4)). But now notice that 6 is just the product 
of these cycles (which contain no elements in common) taken in any order. A 
little thought as we just mentioned shows that this is not unexpected, as we 
prove in our first theorem. 

Theorem 1.1 Every permutation can be expressed as the product of disjoint 
cycles. 
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Proof Consider any permutation 6 £ Sn on a set S, and assume that 6 has k 
cycles where each cycle is of the form (x, 0x, 0^x, . . . , S'^i'^x) for some i 
with 1 < i < k. (Note that since each x G S must be in some cycle, and since 
the cycles are disjoint, we must have 2!f^jm, = n where n is the number of ele- 
ments in S.) When these cycles are multiplied together, we see that each of the 
corresponding permutations affects only those elements contained in the orbit 
(i.e., cycle) it represents. Hence, by multiplying together all of the cycles, each 
element of S will be accounted for with the same result as 6. 

Another way to see this is to consider the effect of 6 on any x G S. The 
resulting element 6x is exactly the same as the image of x under the product of 
all the (disjoint) cycles of 6 since only the cycle containing x will have any 
effect on it. Since both 6 and the product of its cycles have the same effect on 
any x G S, it must be true that equals the product of its cycles. I 

At this point, there is no substitute for simply working out an example for 
yourself. Thus, the reader should pick some permutation, find its cycles, and 
then multiply them together. In so doing, the proof of Theorem 1.1 should 
become quite obvious (or see the exercises at the end of this section). 

Suppose that S = {1, 2, . . . , m} and consider the product of the 2-cycles 
(1, m), (1, m - 1), ... , (1, 3), (1, 2). Expressing these in terms of their corre- 
sponding permutations, we have (note the order of factors since we are multi- 
plying from right to left, and these cycles are not disjoint) 

'1 2 3 4 ••• m\ /I 2 3 4 ••• mVl 2 3 4 ••• m\ 
2 3 4 ••• 1 j '"" 1,3 2 1 4 ••• m]\l 1 3 4 ••• m] 

'1 2 3 4 ••• m\ 
~,2 3 4 5 ••• ij' 

But this last permutation is just the m-cycle (1, 2, . . . , m). A similar calcula- 
tion shows that in fact any m-cycle of the form (ai, aj, . . . , am) may be 
written as the product (ai, am) • • • (Si, a3)(ai, az). (We remark that the multi- 
plication of 2-cycles is one place where multiplying from left to right would 
be more natural.) 

Example 1.5 Consider the permutation 



/I 2 3 4 5 6\ 
1^3 5 2 6 1 4] 
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and its cycle (1, 3, 2, 5). We claim that this cycle may be written as the prod- 
uct (1, 5)(1, 2)(1, 3). There are actually two equivalent ways of seeing this. 
First, we could write out all of the complete permutations as 



(1,3,2,5) = 



'1 2 3 4 5 6^ 
^3 5 2 4 1 6y 



(1,3) = 



'1 2 3 4 5 6^ 
3 2 1 4 5 6 



(1,2) = 



'1 2 3 4 5 6^ 
^2 1 3 4 5 6y 



(1,5) = 



1 2 3 4 5 6] 
5 2 3 4 1 6 



It is then easy to see that (1, 3, 2, 5) = (1, 5)(1, 2)(1, 3). 

On the other hand, we could also leave out those elements in each permu- 
tation that are not affected by any of the cycles, and simply write 



(1,3,2, 5) = 



'1 3 2 5^ 
^3 2 5 1^ 



(1,3) = 



/I 3 2 5^ 



3 12 5; 



(1,2) = 



'1 3 2 5^ 
^2 3 15^ 



(1,5) = 



'1 3 2 5^ 
^5 3 2 1^ 



Again, we obtain (1, 3, 2, 5) = (1, 5)(1, 2)(1, 3). At this point you should be 
sufficiently familiar with the cycle notation to be able to multiply cycles 
without reverting to two-line notation. / 

All of this discussion has shown that any m-cycle may be written as a 
product of 2-cycles, which are usually called transpositions. However we 
could also write, for example, (1, 2, . . . , m) = (m, m - 1) • • • (m, 2)(m, 1) so 
that this decomposition is by no means unique. 

With all of this background, it is now easy to prove an important result in 
the description of permutations. 



Theorem 1.2 Every permutation can be written as the product of transposi- 
tions. 



Proof Theorem 1.1 showed that every permutation can be written as the 
product of disjoint cycles, while we just showed that any cycle can be written 
(in a non-unique manner) as the product of transpositions. I 

In view of this theorem, we say that a permutation is even (odd) if it can 
be written as the product of an even (odd) number of transpositions. Of 
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course, since the decomposition of cycles into transpositions is not unique, we 
must be sure that such a designation is unambiguous. This is the intent of our 
next theorem. 

Theorem 1.3 If a permutation can be represented by an even (odd) number 
of transpositions in one decomposition, then any other decomposition must 
also be an even (odd) number of transpositions. 

Proof Define the polynomial p in n real variables by 



We claim that op = -p. To see this in detail, let a be the transposition (Xa, Xb). 
We assume without loss of generality that Xa < Xb , and write out all of those 
terms in p(x], . . . , xj that contain either Xa or Xb (or both). Thus, those terms 
containing Xa are 





and let a E Sn be any transposition. By op we mean 



Op(Xi, . . . , Xn) = p(Xol , . . . , Xon) • 



|(Xi-xJ(X2-xJ---(x^_l-X, 



a 



a-\ terms 



b-a-\ terms 



)}x{C 




1 term 



n-b terms 



while those containing Xb are 



{(^1 - H )(^2 -H)--- (^a-i - % )} X {(^« - H 
•■ ,, ' ^ ,, — 



)} 



1 1 

a-\ terms already 

counted 



b-a-l terms 
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x {(%-%+l)---(%-^n)} 

n-b terms 



Since all of these terms are multiplied together, we see that if Xa and Xb are 
interchanged (but not Xa+i and Xb+i etc.), there will be no net effect on the 
polynomial p(xj, . . . , Xn) except for the unpaired term (xa - Xb) which results 
in a single change of sign. This shows that op = -p as claimed. 

Now, for any other 6 £ Sn, Theorem 1.2 shows that 9 = HiCTi where each 
Oi is a transposition. Thus, if 

k 

(=1 

we see that 

/ k \ 



Similarly, if 

m 

e.\[o. 

(=1 



we have 0p = (-l)™p. Therefore, if 6 is represented by k transpositions and 

by m transpositions, we must have (-l)''p = (-l)°^p, and hence k and m must 
both be even or both be odd. I 



This result allows us to make the unambiguous definition of the sign of a 
permutation as follows. We define the sign of a permutation 6, sgn 6, by 



sgnO = 



[+1 if is even 
-life is odd ■ 



Our next theorem will be of great benefit to us when we come to discuss the 
theory of determinants in Chapter 4. 

Theorem 1.4 For any two permutations 6, (|) £ Sn we have 



sgn(0(t)) = (sgn 0)(sgn (j)) . 



Proof By Theorem 1.2, we may write 6 as a product of k transpositions and (j) 
as a product of m transpositions. Therefore it follows from Theorem 1.3 that 
sgn e = (-1)'' and sgn (j) = (-1)"". But then 
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sgn(e(|)) = (-1)'^+'^ = (-l)^(-l)'^ = (sgn eXsgn (|)) . I 



As the final topic in our treatment of permutations, we take a look at the 
inverse of a given transposition. For any given transposition (ai, a2), it should 

be obvious that (ai, a2)^ is just the identity transposition. This may be formally 
shown by noting that 



(ai,a2) =(«i,a2)(«i'«2) = 





a A 








,«2 


a J 


,«2 


a J 





Since the identity element in any group is unique, this means that for any 
transposition a we have a"' = a. In view of this result, one might rightfully 
expect that the sign of an inverse permutation is the same as the sign of the 
permutation itself. 

Theorem 1.5 For any 6 £ Sn we have sgn 6"' = sgn 9. 

Proof By Theorem 1.2, we write 6 = 0102 • • • Om where each Oi is a transpo- 
sition. Then, using the fact that 9 is just a product of elements in the group S2 , 
we see that 

9-' = (Oi 02 • • • Om)"' = Om"' • • • 02"' or' = Om • • • O2 Oi 

and hence sgn 9"' = (-1)°^ = sgn 9. I 



Exercises 



1. Consider the following permutations 



6 = 



1 2 3 4^ 
J 4 3 2y 



(t> = 



'1 2 3 4^ 
.3 1 4 2y 



and compute each of the following: 

(a) 9(t) (b) (t)9 (c) 9-> (d) (j)"' 

(e) 9-'(t)-' (f) (t)-'9-' (g) (9(t))-' (h) ((t)9)-' 



2. Referring to Example 1.3, evaluate gfgf gf. How is f related to f? 



3. Find all of the orbits and cycles of the following permutations: 
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'1 2345678 9^ 
^2 3 4 5 1 6 7 9 8y 



(b) 



'1 2 3 4 5 6^ 
^6 5 4 3 1 2J 



4. Express each of the following as the product of disjoint cycles: 

(a) (1, 2, 3)(4,5)(1, 6, 7,8,9X1,5) 

(b) (1,2)(1,2,3)(1,2) 



5. Determine which of the following products of cycles is an even permuta- 
tion: 

(a) (1,2,3)(1,2) 

(b) (1,2,3,4,5)(1,2,3)(4,5) 

(c) (1,2)(1,3)(1,4)(2,5) 

6. Show that the set An C Sn consisting of even permutations forms a group. 
Show that Sn consists of n!/2 even permutations and n!/2 odd permuta- 
tions. 



7. Compute 6 '{j)6 for each of the following: 

(a) = (1,3,5X1,2) (t) = (l,5,7,9). 

(b) 6 = (5, 7, 9) (|) = (1,2,3). 



8. Show that permutations with the same cycle structure belong to the same 
class (see Exercise 1.1.8). In other words, if 6, (|) G Sn, show that 6(j)6"' has 
the same cycle structure as (j). [Hint: Using two-line notation, show that 
6(t)6~' may be evaluated by simply applying 6 to the top and bottom rows 
of (j) separately.] 



9. Show that Sn is non-abelian if n > 3. 
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1.3 HOMOMORPHISMS OF GROUPS 

We now turn our attention to a discussion of mappings from one group to 
another. These results will be absolutely fundamental to everything else that 
follows, and it is essential that the reader thoroughly understand the concepts 
to be presented in this section. 

Let (]): G G' be a mapping from a group G to a group G'. If for every x, 
y £ G we have 

(t)(xy) = (j)(x)(t)(y) 

then (|) is said to be a homomorphism, and the groups G and G' are said to be 
homomorphic. In other words, a homomorphism preserves group multiplica- 
tion, but is not in general either surjective or injective. It should also be noted 
that the product xy is an element of G while the product ^(x) ^(y) is an ele- 
ment of G'. 

Example 1.6 Let G be the (abelian) group of all real numbers under addition, 
and let G' be the group of nonzero real numbers under multiplication. If we 
define (|): G ^ G' by (|)(x) = l"", then 

(|)(x + y) = 2^ +y = 2^ ly = (t)(x) (t)(y) 
so that (j) is indeed a homomorphism. / 

Example 1.7 Let G be the group of all real (or complex) numbers under 
ordinary addition. For any real (or complex) number a, we define the mapping 
(j) of G onto itself by ^(x) = ax. This (j) is clearly a homomorphism since 

(j)(x + y) = a(x + y) = ax + ay = (j)(x) + (j)(y) . 

However, if b is any other nonzero real (or complex) number, then we 
leave it to the reader to easily show that the ("nonhomogeneous") mapping 
ij)(x) = ax + b is not a homomorphism. / 

Let e be the identity element of G, and let e' be the identity element of G'. 
If (|): G ^ G' is a homomorphism, then (|)(x)e' = (|)(x) = (|)(xe) = (|)(x)(|)(e), and 
we have the important result 

m = e'. 

Using this result, we then see that e' = (t)(e) = (t)(xx"') = ^(x) (t)(x"'), and hence 
the uniqueness of the inverse tells us that 
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(|)(x-i) = (|)(x)-i . 

It is very important to note that in general (l)(x)"' (t)~'(x) since if x G G we 
have (t)(x)"' E G' while if x E G', then (t)"'(x) E G. Using these results, it 
should now be easy for the reader to show that (j)(G) forms a subgroup of G' 
(see Exercise 1.3.1). 

In general, there may be many elements x E G that map into the same ele- 
ment x' E G' under (j). It is of particular interest to see what happens if more 
than one element of G (besides e) maps into e'. If k E G is such that (j)(k) = e', 
then for any x E G we have (t)(xk) = (j)(x) (j)(k) = (t)(x)e' = (t)(x). Therefore if 
xk ^ X we see that ^ could not possibly be a one-to-one mapping. To help us 
get a hold on when a homomorphism is one-to-one, we define the kernel of ^ 
to be the set 

Ker (|) = {x E G: (|)(x) = e'} . 

It is also easy to see that Ker (j) is a subgroup of G (see Exercise 1.3.1). 

If a homomorphism (|): G ^ G' is one-to-one (i.e., injective), we say that (j) 
is an isomorphism. If, in addition, ^ is also onto (i.e., surjective), then we say 
that G and G' are isomorphic. In other words, G and G' are isomorphic if ^ is 
a bijective homomorphism. (We point out that many authors use the word 
"isomorphism" to implicitly mean that (j) is a bijection.) In particular, an iso- 
morphism of a group onto itself is called an automorphism. 

From the definition, it appears that there is a relationship between the 
kernel of a homomorphism and whether or not it is an isomorphism. We now 
proceed to show that this is indeed the case. By way of notation, if H is a 
subset of a group G, then by Hg we mean the set Hg = {hg E G: h E H}. 
Recall also that if (j): G ^ G' and x' E G' then, by an inverse image of x', we 
mean any element x E G such that (j)(x) = x'. 

Theorem 1.6 Let (|) be a homomorphism of a group G onto a group G', and 
let K,j, be the kernel of ^. Then given any x' E G', the set of all inverse images 
of x' is given by K,j, x where x E G is any particular inverse image of x'. 

Proof Consider any k E K,j, . Then by definition of homomorphism, we must 
have 

(t)(kx) = (t)(k) (t)(x) = e'x' = x' . 

In other words, if x is any inverse image of x', then so is any kx E K,j, x. We 
must be sure that there is no other element y E G, y ^ K,j, x with the property 
that (t)(y) = x'. 
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To see that this is true, suppose (|)(y) = x' = (|)(x). Then (|)(y) = (|)(x) implies 

that 

e' = (t)(y)(t)(x)-' = (t)(y)(t)(x-') = <^(yx-') . 

But this means that yx"^ £ K,j, , and hence yx"' = k for some k £ K,j, . 
Therefore y = kx £ K,j, x and must have already been taken into account. I 

Corollary A homomorphism (j) mapping a group G to a group G' is an iso- 
morphism if and only if Ker ^ = {e}. 

Proof Note that if (j)(G) ^ G', then we may apply Theorem 1.6 to G and ^(G). 
In other words, it is trivial that (|) always maps G onto ^(G). Now, if (j) is an 
isomorphism, then it is one-to-one by definition, so that there can be no ele- 
ment of G other than e that maps into e'. Conversely, if Ker (j) = {e} then 
Theorem 1.6 shows that any x' £ ^(G) C G' has exactly one inverse image. I 

Of course, if (j) is surjective, then (^(G) is just equal to G'. In other words, 
we may think of isomorphic groups as being essentially identical to each 
other. 

Example 1.8 Let G be any group, and let g G G be fixed. We define the 
mapping (j): G — * G by {j)(x) = gxg"' , and we claim that (j) is an automorphism. 
To see this, first note that (j) is indeed a homomorphism since for any x, y G G 
we have 

^(xy) = g(xy)g~^ = g(xey)g~^ = g(xg~^gy)g~^ = (gxg~^)(gyg~^) 
= (p{x)4){y). 

To see that (j) is surjective, simply note that for any y G G we may define x = 
g 'yg so that (j)(x) = y. Next, we observe that if (j)(x) = gxg"' = e, then right- 
multiplying by g and left multiplying by g"' yields 

X = (g"'g)x(g-'g) = g-'eg = e 

and hence Ker ^ = {e}. From the corollary to Theorem 1.6, we now see that ^ 
must be an isomorphism. / 
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Exercises 



1. 

2. 
3. 

4. 

5. 
6. 

7. 
8. 



Let (j): G — * G' be a homomorphism. 

(a) Show that (j)(G) is a subgroup of G'. 

(b) Show that Ker (|) is a subgroup of G. 

Show that the composition (|)o'ij):A^Cisa homomorphism if both (|): 
B ^ C and ij): A ^ B are. 

Determine which of the following mappings (j): G — * G' are homomor- 
phisms, and for those that are, determine their kernel: 

(a) G = G' = the group of nonzero real numbers under multiplication, and 
(|)(x) = for all X e G. 

(b) Repeat part (a) but with (|)(x) = 1^. 

(c) G = G' = the group of all real numbers under addition, and (|)(x) = 1 + 
x for all X G G. 

(d) Repeat part (c), but with (j)(x) = kx for any (fixed) number k. 

Show that an isomorphism ^ defines an equivalence relation on the set of 
all groups. 

If G is abelian and G' is isomorphic to G, prove that G' is also abelian. 

Let R"^ denote the set of all real numbers > 0, and define the mapping (j): 
R"^ ^ IR by (j)(x) = logio X for each x G R"^. Let R"^ be a group with respect 
to multiplication, and let R be a group with respect to addition. Show that 
(|) is an isomorphism. 

Let A and B be groups (with their own group operations). Show that A x 
B is isomorphic to B x A (see Exercise 1.1.5). 

(a) (Cayley's theorem) Prove that every group G of order n is isomorphic 
to a subgroup of Sn for some S. [Hint: By the rearrangement lemma 
(Exercise 1.1.7), we know that hG = G for any h G G. If G = {gi, . . . , gn}, 
define the mapping ij): G ^ Sn by 



for every a G G. Using the techniques of Exercise 1.2.8, show that ij) is a 
homomorphism, i.e., ip(ab) = i^(a)i[)(b).] 
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(b) Explain why this result shows that there can only be a finite number of 
non-isomorphic groups of order n. 



1.4 RINGS AND FIELDS 



Before starting our discussion of vector spaces, let us first define precisely 
what is meant by a field. We shall see that this is simply a generalization of 
those essential properties of the real and complex numbers that we have been 
using all along. For the sake of completeness and future reference, we will do 
this in a somewhat roundabout manner. 

A nonempty set R together with two operations denoted by + and • is said 
to be an associative ring if it obeys the following axioms for all a, b, c £ R: 



(Rl) a + bGR; 

(R2) a + b = b + a; 

(R3) (a + b) + c = a + (b + c); 

(R4) There exists an element £ R such that a + = a; 
(R5) There exists and element -a £ R such that a + (-a) = 0; 
(R6) a«beR; 
(R7) (a«b)«c = a«(b«c); 

(R8) a«(b + c) = a«b + a«c and (a + b)«c = a«c + b«c. 



Since every ring that we will ever discuss obeys (R7), we henceforth drop 
the adjective "associative" when discussing rings. It should also be noticed 
that (Rl) - (R5) simply require that R be an abelian group under the operation 
+ which we call addition. In addition to these axioms, if there exists an ele- 
ment 1 G R such that a»l = l»a = a for every a G R, then R is said to be a ring 
with unit element. Furthermore, if for every a, b G R we have a»b = b»a, then 
R is called a commutative ring. As usual, we shall generally leave out the 
multiplication sign when dealing with rings. 

Example 1.9 The set Z of all real integers under the usual operations of 
addition and multiplication is a commutative ring with unit element. However, 
the set of even integers under addition and multiplication is a commutative 

ring with no unit element. Note also that the set Z of positive integers is not a 
ring since there are no additive inverse (i.e., negative) elements in this set. / 

Note that while the elements of a ring form an additive abelian group, we 
have not required that each element have a multiplicative inverse. However, if 
the nonzero elements of a ring R happen to form a group under multiplication, 
we say that R is a division ring. In this case we denote the unit element of R 
by 1, and we let a"' denote the inverse of any element a G R. The reason that 
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only the nonzero elements are considered is that has no inverse 0"' such that 
0»0"' = 1. Finally, a field is defined to be a commutative division ring. We 
will generally denote an arbitrary field by the symbol ^. 

Example 1.10 It should be clear that the real numbers IR form a field with the 
usual operations of addition and multiplication. However, the set Z of integers 
does not form a field because for any n £ Z with n^O,we have n"^ = 1/n Z 
(except for n = ±1). 

It is also true that the complex numbers C form a field, but this is slightly 
more difficult to prove. To do so, let us denote a complex number a + ib by 
the ordered pair (a, b) G R x R. Referring to Section 0.6 for motivation, we 
define addition and multiplication on these pairs by 

(a, b) + (c, d) = (a + c, b + d) 
(a, b)(c, d) = (ac - bd, ad + be) 

for all a, b, c, d G R. We claim that the set C consisting of all such ordered 
pairs is a field. Some of the details will be left to the reader to fill in, but we 
will show the important points here. The additive identity element is clearly 
(0, 0), the negative of any (a, b) G C is (-a, -b), and the multiplicative identity 
is (1, 0). Multiplication is commutative since 

(a, b)(c, d) = (ac - bd, ad + be) = (ca - db, cb + da) = (c, d)(a, b) . 

To prove associativity, we have 

(a, b)[(c, d)(e, f)] = (a, b)(ce - df, cf + de) 

= (ace - adf - bcf - bde, acf + ade + bee - bdf) 
= (ac - bd, ad + bc)(e, f) 
= [(a, b)(c,d)](e, f). 

Finally, we show that every (a, b) ^ (0, 0) has an inverse in C. Since a and b 
can not both be 0, we have a^ + b^ > 0. We leave it to the reader to show that 

(a'b)(^\^,^=^] = (l,0). 
\a +b a +b / 

(In the notation of Chapter 0, we see that this is just the statement that zz* = 
|zp implies z"' = z*/|zp.) 

We will be using the fields IR and C almost exclusively in this text. / 
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Since we will be using fields (and therefore rings) as our usual number 
system, let us use the defining relations to prove formally that a ring behaves 
in the manner we are accustomed to and expect. 

Theorem 1.7 Let R be a ring with unit element. Then for all a, b £ R we 
have 

(a) aO = Oa = 0. 

(b) a(-b) = (-a)b = -(ab). 

(c) (-a)(-b) = ab. 

(d) (-l)a = -a. 

(e) (-1)(-1)=1. 

Proof (a) aO = a(0 + 0) (by (R4)) = aO + aO (by (R8)). But R is an additive 
abelian group, so that canceling aO from both sides of this equation says that 
aO = 0. Similarly, we see that Oa = (0 + 0)a = Oa + Oa implies Oa = 0. 

(b) ab + a(-b) = a(b + (-b)) (by (R8)) = aO (by (R5)) = (by (a)). 
Therefore the group property of R shows that a(-b) = -(ab). It is clear that we 
also have (-a)b = -(ab). 

(c) (-a)(-b) = -(a(-b)) (by (b)) = -(-(ab)) (by (b) again). But -(-(ab)) is 
the unique inverse to -(ab), and since we also have ab + (-(ab)) = 0, it follows 
that -(-(ab)) = ab. 

(d) a + (-l)a = la + (-l)a = (1 + (-l))a = Oa = so that (-l)a = -a. 

(e) This follows from (d) using a = -1 since (-1)(-1) = -(-1) = 1. I 

Note the fact that R contains a unit element was actually only required for 
parts (d) and (e) of this theorem. 

Exercises 

1 . Let R be a ring. Prove that a ^ - b^ = (a + b)(a - b) and that (a + b)^ = a^ + 
2ab + b^ for all a, b £ R if and only if R is commutative (where by terms 
of the form a^ we mean aa). 

2. Let F denote the set of all mappings from IR into R. For any f, g E F, we 
define (f + g)(x) = f(x) + g(x) and (fg)(x) = f(x)g(x) for each x E R. In 
other words, f + g and fg are in F. Show that this defines a ring of func- 
tions. 
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3. In the previous problem, show that if we replace the product fg by the 
composition fog then this does not define a ring. 

4. Show that the set Q of all rational numbers forms a field. 

5. Consider the set Z[-v^] = {a + bv5^: a, b £ Z}. We define addition and 
multiplication in Z [v5] by 

(ai + biV^") + (a2 + b2v5') = (ai + + (bi + b2)V^ 

and 

(ai + bi-v^)(a2 + hj^) = (siiSi2 + 2bib2) + (aib2 + bia2)'v5^ . 

Show that the set Z [V5] with these operations forms a ring. Does it form a 
field? 

6. Repeat the previous problem with Q instead of Z. 
1.5 MORE ON GROUPS AND RINGS 

In this section we lay the foundation for future work in our chapter on polyno- 
mials. If the reader has not had much experience with abstract algebra, this 
section may prove somewhat long and difficult on a first reading. Because of 
this, the student should feel free to skim this section now, and return to it only 
when it becomes necessary in later chapters. 

We have seen that fields offer a distinct advantage over rings in that ele- 
ments of the field can be divided (since the field contains the multiplicative 
inverse of each nonzero element). It will be of interest to know how certain 
rings can be "enlarged" to form a field. Rather than treat this problem directly, 
we choose to introduce some additional terminology that will be of use in dis- 
cussing further properties of polynomials. 

In view of the fact that a ring has both addition and multiplication defined 
on it, we make the following definition. Let R and R' be rings. A mapping (j): 
R ^ R' is said to be a ring homomorphism if 

(|)(a + b) = (|)(a) + (|)(b) 

and 

(t)(ab) = m m 

for all a, b e R. We see that 
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m = (|)(0 + a) = (|)(0) + (|)(a) 
and therefore ^(0) = 0. Then we also have 

= (j)(a - a) = (j)(a) + (t)(-a) 
so that adding -(|)(a) to both sides yields 

(t)(-a) = -m . 

While these last two results are the exact analogues of what we found for 
groups, not all of our results can be carried over directly. In particular, it must 
be remembered that every element of a group had an inverse in the group, 
while no such requirement is made on the multiplication in an arbitrary ring 
(recall that a ring in which the nonzero elements form a multiplicative group 
is called a division ring). 

If R is a commutative ring, a nonzero element a £ R is said to be a zero 
divisor if there exists a nonzero element b £ R such that ab = 0. We then say 
that a commutative ring is an integral domain if it contains no zero divisors. 
For example, the ring of integers is an integral domain. 

Example 1.11 Consider the set Z of all integers, and let n G Z"^ be fixed. A 
notation that is frequently used in algebra is to write a|b to mean "a divides b," 
(i.e., in this case, that b is an integral multiple of a) and c|'d to mean "c does 
not divide d." We define a relationship between the integers a and b by 
writing 

a = b(mod n) 

if n|(a - b). This relation is called congruence modulo n, and we read it as "a 
is congruent to b modulo n." We leave it as an exercise for the reader to show 
that this defines an equivalence relation on the set of integers (see Exercise 
1.5.2). For example, it should be clear that 5 = 2(mod 3), 23 = 5(mod 6) and 
21 = -9(mod 10). 

Now suppose we define a ring R to be the set of integers mod 6 (this ring 
is usually denoted by Zs). Then the elements of R are the equivalence classes 
of the integers, and we denote the equivalence class of an integer n by [n]. 
Then the elements of R are [0], [1], [2], [3], [4] and [5]. For example, from the 
previous paragraph we see that [5] = [23] because 6|(23 - 5). 

We define addition in R by [a] + [b] = [a + b], and thus [0] is the zero ele- 
ment of R. Defining multiplication in R by [a][b] = [ab], we see, for example. 
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that [2] [5] = [4]. However, note that [2] [3] = [0] even though [2] ^ [0] and 
[3] [0], and thus R is not an integral domain. We will have much more to 
say about this ring in Section 6.6. / 

It should now be clear that arbitrary rings can have a number of properties 
that we generally find rather unpleasant. Another type of pathology that is 
worth pointing out is the following. Let D be an integral domain. We say that 
D is of finite characteristic if there exists some integer m > and some 
nonzero a G D such that ma = 0. Then the smallest positive integer p such that 
pa = for some nonzero a E D is called the characteristic of D. 

If D is an integral domain of characteristic p, then there exists a nonzero 
element a E D such that pa = 0. Then for any x E D we also have 

= (pa)x = (a + • • • + a)x = ax + • • • + ax = a(x + • • • + x) = a(px) . 

But D has no zero divisors, and hence we must have px = for every x G D. If 
D has a unit element , then an equivalent requirement is to say that if D is of 
characteristic p, then 1 + • • • + 1 = 0, where there are p terms in the sum. 
Furthermore, any such sum consisting of less than p terms is nonzero. 

Obviously, the most important types of integral domain for our purposes 
are those of characteristic 0. In other words, to say that D is of characteristic 
means that if m is an integer and a G D is nonzero, then ma = if and only if 
m = 0. The reason that we even bother to mention this is because most of the 
theory of matrices and determinants that we shall develop is valid over an 
arbitrary field ^. For example, we shall obtain results such as det A = -det A 
which implies that 2 det A = 0. However, if ^ happens to be of characteristic 
2, then we can not conclude from this that det A = 0. In this book, we will 
always assume that our fields are of characteristic (except in Section 6.6). 

Returning to our general discussion, let 1 and 1' be the multiplicative 
identities of the rings R and R' respectively, and consider any ring homo- 
morphism (|): R ^ R'. Then 

(t)(a) = (t)(la) = (t)(l)(t)(a) 

but this does not in general imply that (j)(l) = 1'. However, if R' is an integral 
domain and (|)(a) ^ 0, then we have 

= (|)(a)-(|)(l)(|)(a) = (|)(a)[l'-(|)(l)] 

and hence (j)(l) = 1' (note that we do not distinguish in our notation between 
and 0'). 
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As was the case with groups, we define the kernel of (|) to be the set 
Ker (j) = {a E R: (^(a) = 0} . 

If a, b E Ker (j) then 

(|)(a + b) = (|)(a) + (|)(b) = + = 

so that a + b E Ker (|) also. Furthermore, if a E Ker (|) then 

(t)(-a) = -m = 

so that the (additive) inverse of a is also in Ker (|). Thus Ker (|) forms a sub- 
group of R under addition. 

As we also did with groups, we say that a ring homomorphism of R into 
R' is a (ring) isomorphism if it is an injective (i.e., one-to-one) mapping. If 
there exists a bijective ring homomorphism of R onto R', then we say that R 
and R' are isomorphic. Theorem 1.6 also carries over directly to the present 
case, and we then have that a ring homomorphism is an isomorphism if and 
only if Ker (|) = {0}. 

Now note that another very important property of Ker (|) comes from the 
observation that if a E Ker (j) and r E R, then 

(|)(ar) = (l)(a) (t)(r) = 0(t)(r) = . 

Similarly (|)(ra) = 0, and therefore both ar and ra are in Ker (|). We take this 
property as the prototype of a new object defined as follows. 

A nonempty subset I of a ring R is said to be a (two-sided) ideal of R if I 
is a subgroup of R under addition, and if ar E I and ra E I for a// a E I and all 
r E R. It is important to realize that the element r can be any element of R, not 
just an element of I. 

Now let R be a commutative ring with unit element, and let a E R be arbi- 
trary. We denote by (a) the set of all multiples of a by elements of R. (While 
this is a somewhat confusing notation, it nevertheless conforms to standard 
usage.) In other words, 

(a) = {ra:rER} . 

We claim that (a) is actually an ideal of R. Indeed, if r, s E R, then ra, sa E (a) 
and therefore ra + sa = (r + s)a E (a). Next, we have = Oa E (a), and finally, 
the negative (i.e., additive inverse) of ra E (a) is (-r)a which is also in (a). 
This shows that (a) is a subgroup of R under addition. Lastly, for any ra E (a) 
and any s E R, we see that (ra)s = s(ra) = (sr)a E (a). We have thus shown that 
(a) is an ideal. In general, any ideal of the form (a) is called a principal ideal. 
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and the element a £ R is called a generator of (a). A principal ideal (a) is thus 
the smallest ideal of R that contains a. 

Example 1.12 We show that any field ^ has no ideals other than (0) and J^. 
Since the ideal (0) is quite trivial, let I be an ideal and assume that I ^ (0). If 
a £ I, a 9i 0, then a £ ^ implies that a"' £ ^ so that 1 = aa"' £ I by the defini- 
tion of ideal. But now, for any r £ ^ we have r = Ir £ I (again by the 
definition of ideal), and hence I = ^. / 

The converse of this example is given in the next theorem. Recall that a 
field is a commutative division ring, and hence a commutative ring R with 
unit element 1 is a field if every nonzero a £ R has an inverse b £ R with ab = 
1. 

Theorem 1.8 If R is a commutative ring with unit element whose only ideals 
are (0) and R, then R is a field. 

Proof Part of this was proved in the above discussion, but for the sake of 
completeness we repeat it here. Let a £ R be nonzero, and consider the set 

Ra = {ra:reR} . 

We shall first show that this set is an ideal of R. To see this, suppose x, y E 
Ra. Then there exist ri, r2 G R such that x = x^o. and y = rza. But then (using the 
definition of a ring) we see that 

X + y = ria + rja = (ri + r2)a G Ra . 

Next we note that 

-X = -ria = (-ri)a G Ra 

and therefore Ra is a subgroup of R under addition. Now, given any r G R we 
have 

rx = r(ria) = (rri)a E Ra 

and since R is commutative, it also follows that xr E Ra. This shows that Ra is 
an ideal of R. 

By hypothesis, we see that Ra must equal either (0) or R. Since R is a ring 
with unit element, we have Q ^ 2i= laGRa and hence Ra ^ (0). This means 
that we must have Ra = R so that every element of R is a multiple of a. In par- 
ticular, since 1 G R, there must exist an element b E R with the property that 
ba = 1. In other words, b = a"^ and thus R is a field. I 
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Now let H be a subgroup of a group G, and let a £ G be arbitrary. Then 
the set 

Ha = {ha: h E H} 

is called a right coset of H in G. Let a, b E G be arbitrary, and suppose that 
the cosets Ha and Hb have an element in common. This means that hja = hjb 
for some hi, hj E H. But then using the fact that H is a subgroup, we see that 

a = hr'hia = hr'h2b e Hb . 

Since this means that a = hb for some h = hi'^hz E H, we see (using the 
rearrangement lemma of Exercise 1.1.7) that this implies 

Ha = Hhb = Hb 

and therefore if any two right cosets have an element in common, then they 
must in fact be identical. It is easy to see that the set of all right cosets of H in 
G defines a partition of G and hence an equivalence relation that decomposes 
G into disjoint subsets (see Exercise 1.5.15). 

Recall that o(G) denotes the order of G (i.e., the number of elements in the 
group G). We claim that if H is a subgroup of G, then o(H) = o(Ha) for any a 
E G. Indeed, to prove this we show that there is a bijection of H to Ha. Define 
the mapping a: H ^ Ha by a(h) = ha. This is clearly a surjective mapping 
since Ha consists precisely of elements of the form ha for h E H. To see that it 
is also injective, suppose that for some hi, hj E H we have a(hi) = a(h2) or, 
equivalently, h,a = hja. Multiplying from the right by a"' then implies that 
hi = h2, thus showing that a is one-to-one. 

In the particular case of finite groups, the previous paragraph shows that 
any two right cosets of H in G must have the same number o(H) of elements. 
We also showed above that any two distinct right cosets have no elements in 
common. It then follows that any a E G is in the unique right coset Ha, and 
therefore the set of all right cosets of H in G must contain every element of G. 
This means that if there are k distinct right cosets of H in G, then we must 
have ko(H) = o(G) (i.e., o(H)|o(G)), and hence we have proved Lagrange's 
theorem: 

Theorem 1.9 If G is a finite group and H is a subgroup of G, then o(H) is a 
divisor of o(G). 
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The number o(G)/o(H) will be denoted by ioCH), and is usually called the 
index of H in G. (This is frequently denoted by [G : H].) The index of H in G 
is thus the number of distinct right cosets of H in G. 

While we have restricted our discussion to right cosets, it is clear that 
everything could be repeated using left cosets defined in the obvious way. It 
should also be clear that for a general subgroup H of a group G, we need not 
have Ha = aH for any a £ G. However, if N is a subgroup of G such that for 
every n £ N and g G G we have gng"' G N, then we say that N is a normal 
subgroup of G. An equivalent way of phrasing this is to say that N is a normal 
subgroup of G if and only if gNg"' C N for all g G G (where by gNg"' we 
mean the set of all gng"' with n G N). 

Theorem 1.10 A subgroup N of G is normal if and only if gNg"' = N for 
every g G G. 

Proof If gNg"' = N for every g G G, then clearly gNg"' C N so that N is nor- 
mal. Conversely, suppose that N is normal in G. Then, for each g G G we 
have gNg"' C N, and hence 

g"'Ng = g"'N(g"')"' C N . 

Using this result, we see that 

N = (gg"')N(gg"') = g(g"'Ng)g"' C gNg"' 

and therefore N = gNg"' (This also follows from Example 1.8). I 

The reader should be careful to note that this theorem does not say that 
gng"' = n for every n G N and g G G. This will in general not be true. The use- 
fulness of this theorem is that it allows us to prove the following result. 

Theorem 1.11 A subgroup N of G is normal if and only if every left coset of 
N in G is also a right coset of N in G. 

Proof If N is normal, then gNg"' = N for every g G G, and hence gN = Ng. 
Conversely, suppose that every left coset gN is also a right coset. We show 
that in fact this right coset must be Ng. Since N is a subgroup it must contain 
the identity element e, and therefore g = ge G gN so that g must also be in 
whatever right coset it is that is identical to gN. But we also have eg = g so 
that g is in the right coset Ng. Then, since any two right cosets with an ele- 
ment in common must be identical, it follows that gN = Ng. Thus, we see that 
gNg"' = Ngg"' = N so that N is normal. I 
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If G is a group and A, B are subsets of G, we define the set 

AB = {ab E G: a E A, b E B} . 

In particular, if H is a subgroup of G, then HH C H since H is closed under the 
group multiplication operation. But we also have H = He C HH (since e E H), 
and hence HH = H. 

Now let N be a normal subgroup of G. By Theorem 1 . 1 1 we then see that 

(Na)(Nb) = N(aN)b = N(Na)b = NNab = Nab . 

In other words, the product of right cosets of a normal subgroup is again a 
right coset. This closure property suggests that there may be a way to 
construct a group out of the cosets Na where a is any element of G. We now 
show that there is indeed a way to construct such a group. Our method is used 
frequently throughout mathematics, and entails forming what is called a 
quotient structure. 

Let G/N denote the collection of all right cosets of N in G. In other words, 
an element of G/N is a right coset of N in G. We use the product of subsets as 
defined above to define a product on G/N. 

Theorem 1.12 Let N be a normal subgroup of a group G. Then G/N is a 
group. 

Proof We show that the product in G/N obeys properties (Gl) - (G4) in the 
definition of a group. 

(1) If A, B E G/N, then A = Na and B = Nb for some a, b E G, and hence 
(since ab E G) 

AB = NaNb = Nab E G/N . 

(2) If A, B, C E G/N, then A = Na, B = Nb and C = Nc for some a, b, c E 
G and hence 

(AB)C = (NaNb)Nc = (Nab)Nc = N(abN)c = N(Nab)c = N(ab)c 
= Na(bc) = Na(Nbc) = Na(NbNc) = A(BC). 

(3) If A = NaEG/N,then 

AN = NaNe = Nae = Na = A 

and similarly 

NA = NeNa = Nea = Na = A . 
Thus N = Ne E G/N serves as the identity element in G/N. 
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(4) If Na e G/N, then Na ^ is also in G/N, and we have 

NaNa"' = Naa"' = Ne 

as well as 

Na-^Na = Na'^a = Ne . 

Therefore Na"' £ G/N is the inverse to any element Na £ G/N. I 

Corollary If N is a normal subgroup of a finite group G, then o(G/N) = 
o(G)/o(N). 

Proof By construction, G/N consists of all the right cosets of N in G, and 
since this number is just the definition of iaCN), we see that o(G/N) = 
o(G)/o(N). I 

The group defined in Theorem 1.12 is called the quotient group (or 
factor group) of G by N. 

Let us now apply this quotient structure formalism to rings. Since any sub- 
group of an abelian group is automatically normal, and since any ring R is an 
abelian group under addition, any ideal I of R is therefore a normal subgroup 
of R (under addition). It is clear that we can now form the quotient group R/I 
where the elements of R/I are the cosets of I in R (since R is abelian, there is 
no need to distinguish between right and left cosets). We write these cosets as 
I + r (or r + 1) for each r £ R. In the next theorem we show that R/I can in fact 
be made into a ring which is called the quotient ring of R by I. 

Theorem 1.13 Let I be an ideal of a ring R. For any I + a, I + b G R/I we 

define 

(I + a) + (I + b) = I + (a + b) 

and 

(I + a)(I + b) = I + ab . 
Then, with these operations, R/I forms a ring. 

Proof From the proof of Theorem 1.12, it is obvious that R/I forms a group 
under addition if we use the composition rule (I + a) + (I + b) = I + (a + b) for 
all a, b £ R. We now turn our attention to the multiplication rule on R/I, and 

we begin by showing that this rule is well-defined. In other words, we must 
show that if I + a = I + a' and 1 + b = 1 + b', then I + ab = I + a'b'. From I + a = 
I + a', we have a = x + a' for some x E I, and similarly b = y + b' for some y E 
I. Then 
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ab = (x + a')(y + b') = xy + xb' + a'y + a'b' . 

But I is an ideal so that xy, xb', and a'y are all elements of I, and hence z = 
xy + xb' + a'y E I. Therefore, ab = z + a'b' so that 

ab + 1 = a'b' + z + 1 = a'b' + 1 

as desired. 

To show that R/I is a ring, we must verify that the properties (Rl) - (R8) 
given in Section 1.4 hold in R/I. This is straightforward to do, and we give one 
example, leaving the rest to the reader (Exercise 1.5.5). To prove the first part 
of (R8), suppose a, b, c E R. Then I + a, I + b, I + cE R/I and hence 

(I + a)[a + b) + a + c)] = (I + a)[I + (b + c)] 

= I + a(b + c) 
= I + (ab + ac) 
= (I + ab) + (I + ac) 
= (I + a)(I + b) + (I + a)(I + c). 

Example 1.13 Recall that the set Z of all integers forms a commutative ring 
with unit element (see Example 1.9). If we choose any n G Z, then n generates 
a principal ideal (n) that consists of all numbers of the form na for each a G Z. 
For example, the number 2 generates the principal ideal (2) that is nothing 
more than the ring of all even integers. The quotient ring Z/(2) is then the set 
of all cosets of (2). Each of these cosets is either the set of even integers, or 
the even integers plus some odd integer. / 

We have now finished essentially all of the mathematical formalism nec- 
essary to undertake a rigorous study of linear algebra. In the next chapter we 
begin our treatment of the subject matter proper of this text. 

Exercises 

1. Let (|) be a homomorphism of a group G into a group G', and let K,j, be the 
kernel of (|). Prove that K,j, is a normal subgroup of G. 

2. This exercise refers to the relation "congruence modulo n" defined in 
Example 1.11. Throughout this exercise, let n G Z"^ be arbitrary but fixed. 

(a) Show that this relation defines an equivalence relation. 

(b) Using Theorem 0.8 to divide a by n, show that the congruence rela- 
tion has exactly n distinct equivalence sets. 
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(c) If a = b(mod n) and c = d(mod n), show that a + c = (b + d)(mod n) 
and ac = bd(mod n). 

(d) Show Z/(n) is isomorphic to the integers mod n. 

3. Let (|): R -» R' be a ring isomorphism. Show that R' is commutative if R 
is. 

4. Let (|): R ^ R' be a ring isomorphism. Show that R' is an integral domain 
if Ris. 

5. Finish the proof that R/I forms a ring in Theorem L 13. 

6. Prove that an integral domain is a field if and only if every nonzero ele- 
ment has a multiplicative inverse. 

7. Show that the kernel of a ring homomorphism is an ideal. 

8. Determine all the subgroups of the permutation group S3. Which of these 
is normal? 

9. Let fA/^ be a collection of normal subgroups of a group G. Show that the 
intersection of all N E fA^is a normal subgroup of G. 

10. Prove or disprove the following statement: If (|): R ^ R' is a ring homo- 
morphism, then the image of (|) is an ideal of R'. 

11. Let (j) be a homomorphism of a group G onto a group G', and let K be the 
kernel of (]). By Exercise 5.1, we know that K is a normal subgroup of G, 
and hence we may form the quotient group G/K. Prove that G/K is iso- 
morphic to G'. [Hint. Since any element in X E G/K is of the form Kg 
where g E G, define the mapping -ip: G/K G' by il)(X) = il)(Kg) = (|)(g). 
To show that is an isomorphism, first show that ip is well-defined, that 
is, X = Kg = Kg' implies (j)(g) = <^(g'). Next, show that is a homomor- 
phism, i.e., that ip(XY) = ip(X) ![)(¥). Now show that is surjective (use 
the fact that (j) is surjective). Finally, show that Ker ip = {0} (you will 
need the additional fact that the identity in G/K is K = Ke).] 

12. Show that a field ^ can have no zero divisors. 

13. Let H be a subgroup of a group G. Show that the set of all right cosets of 
H in G decomposes G into disjoint subsets. 
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14. The center of a group G is the set Z = {z £ G: zg = gz for all g £ G}. 
Show that Z is a normal subgroup of G. 

15. Show that the set {0, 1} with the usual addition and multiplication opera- 
tions, but subject to 1 + 1 = 0, forms a field of characteristic 2. (This is an 
example of a finite field.) 



16. Let G be a group and let Gi, G2 be subgroups of G with Gi H G2 = e. 
Show that Gi and G2 commute if and only if they are normal subgroups. 



CHAPTER 2 



Vector Spaces 



We now begin our treatment of the principal subject matter of this text. We 
shall see that all of linear algebra is essentially a study of various transforma- 
tion properties defined on a vector space, and hence it is only natural that we 
carefully define vector spaces. This chapter therefore presents a fairly rigorous 
development of (finite-dimensional) vector spaces, and a discussion of their 
most important fundamental properties. Basically, the general definition of a 
vector space is simply an axiomatization of the elementary properties of ordi- 
nary three-dimensional Euclidean space. 

2.1 DEFINITIONS 

A nonempty set V is said to be a vector space over a field ^ if: (i) there exists 
an operation called addition that associates to each pair x, y G V a new vector 
X + y G V called the sum of x and y; (ii) there exists an operation called 
scalar multiplication that associates to each a G ^ and x G V a new vector 
ax G V called the product of a and x; (iii) these operations satisfy the 
following axioms: 

(VI) X + y = y + X for all X, y G V. 

(V2) (x + y) + z = X + (y + z) for all x, y, z G V. 

(V3) There exists an element G V such that + x = x for all x G V. 
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(V4) For all X e V there exists an element -x E V such that x + (-x) = 0. 

(V5) a(x + y) = ax + ay for all x, y G V and all a G ^. 

(V6) (a + b)x = ax + bx for all x G V and all a, b G ^. 

(V7) a(bx) = (ab)x for all x G V and all a, b G ^. 

(V8) Ix = X for all X G V where 1 is the (multiplicative) identity in J^. 

Note that (VI) - (V4) simply require that V be an additive abelian group. 
The members of V are called vectors, and the members of ^ are called 
scalars. The vector G V is called the zero vector, and the vector -x is called 
the negative of the vector x. 

We mention only in passing that if we replace the field ^ by an arbitrary 
ring R, then we obtain what is called an R-module (or simply a module over 
R). If R is a ring with unit element, then the module is called a unital R- 
module. In fact, this is the only kind of module that is usually considered in 
treatments of linear algebra. We shall not discuss modules in this text, 
although the interested reader can learn something about them from several of 
the books listed in the bibliography. 

Throughout this chapter V will always denote a vector space, and the cor- 
responding field ^ will be understood even if it is not explicitly mentioned. If 
^ is the real field IR, then we obtain a real vector space while if ^ is the com- 
plex field C, then we obtain a complex vector space. It may be easiest for the 
reader to first think in terms of these spaces rather than the more abstract 
general case. 

Example 2.1 Probably the best known example of a vector space is the set 

= ^ X • • • X ^ of all n-tuples (ai, . . . , an) where each ai G J^. To make 
into a vector space, we define the sum of two elements (ai, . . . , an) G JF" and 
(bi,...,bn)GjF"by 

(ai, . . . , an) + (bi, . . . , bn) = (ai + bi, . . . , an + bn) 
and scalar multiplication for any k G ^ by 

k(ai, . . . , an) = (kai, . . . , kan) . 

If A = (ai, . . . , an) and B = (bi, . . . , bn), then we say that A = B if and only if 
ai = bi for each i = 1, . . . , n. Defining = (0, . . . , 0) and -A = (-ai, . . . , -an) 

as the identity and inverse elements respectively of the reader should have 
no trouble verifying properties (VI) - (V8). 

The most common examples of the space come from considering the 
fields IR and C. For instance, the space IR'^ is (with the Pythagorean notion of 
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distance defined on it) just the ordinary three-dimensional Euclidean space 
(x, y, z) of elementary physics and geometry. 

We shall soon see that any finite-dimensional vector space V over a field 
f is essentially the same as the space In particular, we will prove that V is 
isomorphic to jF" for some positive integer n. / 

Example 2.2 Another very useful vector space is the space ^[x] of all poly- 
nomials in the indeterminate x over the field J (polynomials will be defined 
carefully in Chapter 6). In other words, every element in ^[x] is a polynomial 

of the form ao + ajX + • • • + anX° where each ai £ ^ and n is any positive inte- 
ger (called the degree of the polynomial). Addition and scalar multiplication 
are defined in the obvious way by 

n n n 

;=0 (=0 j=0 



and 



^E"'^' =E(^"')^'• 



^•=0 



(=0 



(If we wish to add together two polynomials 2f=o SjX^ and I]P=o biX^ where 
m > n, then we simply define ai = for i = n + 1, . . . , m.) 

Since we have not yet defined the multiplication of vectors, we ignore the 
fact that polynomials can be multiplied together. It should be clear that ^[x] 
does indeed form a vector space. / 

Example 2.3 We can also view the field C as a vector space over R. In fact, 
we may generally consider the set of n-tuples (xi, . . . , Xn), where each Xj G C, 
to be a vector space over IR by defining addition and scalar multiplication (by 
real numbers) as in Example 2.1. We thus obtain a real vector space that is 
quite distinct from the space C°. / 

We now prove several useful properties of vector spaces that are analo- 
gous to the properties given in Theorem 1.7 for rings. 

Theorem 2.1 Let V be a vector space over ^. Then for all x, y, z G V and 
every a G ^ we have 

(a) X + y = z + y implies x = z. 

(b) ax = if and only if a = or x = 0. 

(c) -(ax) = (-a)x = a(-x). 
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Proof We first remark that there is a certain amount of sloppiness in our 
notation since the symbol is used both as an element of V and as an element 
of f. However, there should never be any confusion as to which of these sets 
lies in, and we will continue with this common practice. 

(a) If X + y = z + y, then 

(x + y) + (-y) = (z + y) + (-y) 

implies 

X + (y + (-y)) = z + (y + (-y)) 

which implies x + = z + and hence x = z. This is frequently called the 
(right) cancellation law. It is also clear that x + y = x + z implies y = z (left 
cancellation). (This is just a special case of the general result proved for 
groups in Section 1.1.) 

(b) If a = 0, then 

Ox = (0 + 0)x = Ox + Ox . 

But Ox = + Ox so that + Ox = Ox + Ox, and hence (a) implies = Ox. If x = 
0, then 

aO = a(0 + 0) = aO + aO . 

But aO = + aO so that + aO = aO + aO, and again we have = aO. 
Conversely, assume that ax = 0. If a then a'' exists, and hence 

X = Ix = (a"'a)x = a"'(ax) = a"'0 = 

by the previous paragraph. 

(c) By (V4) we have ax + (-(ax)) = 0, whereas by (b) and (V6), we have 

= Ox = (a + (-a))x = ax + (-a)x . 

Hence ax + (-(ax)) = ax + (-a)x implies -(ax) = (-a)x by (a). Similarly, = 
X + (-x) so that 

= aO = a(x + (-x)) = ax + a(-x) . 
Then = ax + (-(ax)) = ax + a(-x) implies -(ax) = a(-x). I 

In view of this theorem, it makes sense to define subtraction in V by 

x-y = x + (-y) . 

It should then be clear that a vector space will also have the properties we 
expect, such as a(x - y) = ax - ay, and -(x - y) = -x + y. 
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If we take an arbitrary subset of vectors in a vector space then, in general, 
this subset will not be a vector space itself. The reason for this is that in gen- 
eral, even the addition of two vectors in the subset will not result in a vector 
that is again a member of the subset. Because of this, we make the following 
definition. Suppose V is a vector space over ^ and W C V. Then if x, y E W 
and c E ^ implies x + y E W and cx E W, we say that W is a subspace of V. 
Indeed, if c = then = Ox e W so that e W, and similarly -x = (-l)x e W 
so that -X G W also. It is now easy to see that W obeys (VI) - (V8) if V does. 
It should also be clear that an equivalent way to define a subspace is to require 
that cx + y G W for all x, y G W and all c G ^. 

If W is a subspace of V and W ^Y, then W is called a proper subspace of 
V. In particular, W = {0} is a subspace of V, but it is not very interesting, and 
hence from now on we assume that any proper subspace contains more than 
simply the zero vector. (One sometimes refers to {0} and V as trivial sub- 
spaces of V.) 

Example 2.4 Consider the elementary Euclidean space consisting of all 
triples (x, y, z) of scalars. If we restrict our consideration to those vectors of 

the form (x, y, 0), then we obtain a subspace of R^. In fact, this subspace is 
essentially just the space which we think of as the usual xy-plane. We 
leave it as a simple exercise for the reader to show that this does indeed define 
a subspace of IR^. Note that any other plane parallel to the xy-plane is not a 
subspace. / 

Example 2.5 Let V be a vector space over ^, and let S = {x,, . . . , Xn} be 
any n vectors in V. Given any set of scalars {ai, . . . , an}, the vector 

is called a linear combination of the n vectors Xj G S, and the set S of all such 
linear combinations of elements in S is called the subspace spanned (or gen- 
erated) by S. Indeed, if A = 2 f =i ajXi and B = 2 f =i biXj are vectors in S and 
c e J, then both 

and 

n 

cA = 2(ca;)X; 
(=1 

are vectors in S. Hence 5 is a subspace of V. S is sometimes called the linear 
span of S, and we say that S spans S. II 
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In view of this example, we might ask whether or not every vector space is 
in fact the linear span of some set of vectors in the space. In the next section 
we shall show that this leads naturally to the concept of the dimension of a 
vector space. 



Exercises 

1. Verify axioms (VI) - (V8) for the space 

2. Let S be any set, and consider the collection V of all mappings f of S into 
a field ^. For any f , g £ V and a G JF, we define (f + g)(x) = f(x) + g(x) 
and (af)(x) = af(x) for every x G S. Show that V together with these 
operations defines a vector space over ^. 

3. Consider the two element set {x, y} with addition and scalar multiplica- 
tion by c E defined by 

x + x = x x + y = y + x = y y + y = x cx = x cy = x. 

Does this define a vector space over ^? 

4. Let V be a vector space over J^. Show that if x E V and a, b E ^ with a ^ 
b, then ax = bx implies that x = 0. 

5. Let (V, +, •) be a real vector space with the addition operation denoted by 
+ and the scalar multiplication operation denoted by •. Let Vq E V be 

fixed. We define a new addition operation © by x ® y = x + y + Vq, and 
a new scalar multiplication operation ® by a ® x = a»x + (a - l)»Vo. 
Show that (V, © , ® ) defines a real vector space. 

6. Let F[IR] denote the space of all real- valued functions defined on IR with 
addition and scalar multiplication defined as in Exercise 1.2. In other 
words, f E F[R] means f: R ^ R. 

(a) Let C[R] denote the set of all continuous real-valued functions 
defined on R. Show that C[R] is a subspace of F[R]. 

(b) Repeat part (a) with the set D[R] of all such differentiable functions. 

7. Referring to the previous exercise, let D°[IR] denote the set of all n-times 

differentiable functions from IR to IR. Consider the subset V of D°[IR] 
given by the set of all functions that satisfy the differential equation 
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+ a„_i/("-i)(x) + + • • • + + a,f(x) = 

where f*^^^(x) denotes the ith derivative of f(x) and ai is a fixed real con- 
stant. Show that V is a vector space. 

8. Let V = R^. In each of the following cases, determine whether or not the 
subset W is a subspace of V: 

(a) W = {(x, y, 0): x, y e R} (see Example 2.4). 

(b) W = {(x, y, z) e R^: z > 0}. 

(c) W = {(x, y, z) e R^: x^ + y^ + < 1}. 

(d) W = {(x, y, z) e R^: X + y + z = 0}. 

(e) W = {(X, y, z) e r3: x, y, z e Q}. 

(f) W = {(x,y,z)eR3- {0,0,0}}. 

9. Let S be a nonempty subset of a vector space V. In Example 2.5 we 
showed that the linear span 5 of S is a subspace of V. Show that if W is 
any other subspace of V containing S, then 5 CW. 

10. (a) Determine whether or not the intersection fl "}= i Wi of a finite number 
of subspaces Wi of a vector space V is a subspace of V. 

(b) Determine whether or not the union Uf =i Wi of a finite number of 
subspaces Wi of a space V is a subspace of V. 

11. Let Wi and W2 be subspaces of a space V such that Wi U W2 is also a 
subspace of V. Show that one of the Wi is subset of the other. 

12. Let Wi and W2 be subspaces of a vector space V. If for every v G V we 
have V = wi + W2 where Wi G Wi, then we write V = W] + W2 and say 
that V is the sum of the subspaces Wi. If V = Wi + W2 and Wi H W2 = 
{0}, show that every v G V has a unique representation v = wi + W2 with 
Wi G Wi. 

13. Let V be the set of all (infinite) real sequences. In other words, any v G V 
is of the form (x,, X2, X3 , . . . ) where each Xj G R. If we define the addi- 
tion and scalar multiplication of distinct sequences componentwise 
exactly as in Example 2.1, then it should be clear that V is a vector space 
over R. Determine whether or not each of the following subsets of V in 
fact forms a subspace of V: 

(a) All sequences containing only a finite number of nonzero terms. 



2.1 DEFINITIONS 



75 



(b) All sequences of the form {xi, X2, . . . , xn, 0, 0, . . . } where N is 
fixed. 

(c) All decreasing sequences, i.e., sequences where x^+i < x^ for each 
k=l,2,.... 

(d) All convergent sequences, i.e., sequences for which lim k^oo exists. 

14. For which value of k will the vector v = (1, -2, k) £ IR^ be a linear com- 
bination of the vectors Xi = (3, 0, -2) and Xj = (2, -1, -5)? 

15. Write the vector v = (1, -2, 5) as a linear combination of the vectors xi = 
(1,1, 1),X2 = (1,2, 3)andx3 = (2,-l, 1). 

2.2 LINEAR INDEPENDENCE AND BASES 

Let Xi, . . . , Xn be vectors in a vector space V. We say that these vectors are 
linearly dependent if there exist scalars a i, . . . , a n ^ ^, not all equal to 0, 
such that 



The vectors Xj are said to be linearly independent if they are not linearly 
dependent. From these definitions, it follows that any set containing a linearly 
dependent subset must be linearly dependent, and any subset of a linearly 
independent set is necessarily linearly independent. 

It is important to realize that a set of vectors may be linearly dependent 
with respect to one field, but independent with respect to another. For exam- 
ple, the set C of all complex numbers is itself a vector space over either the 
field of real numbers or over the field of complex numbers. However, the set 
{xi = 1, X2 = i} is linearly independent if ^ = IR, but linearly dependent if ^ = 
C since ixi + (-l)x2 = 0. We will always assume that a linear combination is 
taken with respect to the same field that V is defined over. 

As a means of simplifying our notation, we will frequently leave off the 
limits of a sum when there is no possibility of ambiguity. Thus, if we are con- 
sidering the set {xi, . . . , Xn}, then a linear combination of the Xj will often be 
written as SajXi rather than 2?=i aiXi. In addition, we will often denote a col- 
lection {xi, . . . , Xn} of vectors simply by {xi}. 

Example 2.6 Consider the three vectors in IR-^ given by 



n 
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e,=(l, 0, 0) 
^2 = (0, 1, 0) 
^3 = (0, 0, 1). 



Using the definitions of addition and scalar multiplication given in Example 
2.1, it is easy to see that these three vectors are linearly independent. This is 
because the zero vector in is given by (0, 0, 0), and hence 



are linearly dependent since X3 = xi + 3x2. / 

Theorem 2.2 A finite set S of vectors in a space V is linearly dependent if 
and only if one vector in the set is a linear combination of the others. In other 
words, S is linearly dependent if one vector in S is in the subspace spanned by 
the remaining vectors in S. 

Proof If S = {xi, . . . , Xji} is a linearly dependent subset of V, then 



for some set of scalars a], . . . , an G not all equal to 0. Suppose, to be spe- 
cific, that ai ^ 0. Then we may write 



ai ei + 32 62 + as 63 = (ai, a2, as) = (0, 0, 0) 



implies that ai = a2 = as = 0. 
On the other hand, the vectors 



xi = (1, 0, 0) 
X2 = (0, 1, 2) 
= (1, 3, 6) 



aiXi + a2X2 + • • • + anXn = 



xi = -(a2/ai)x2 



(a n/ai)x, 



■n 



which shows that Xi is a linear combination of x 
Conversely, if = Sj^ia^-jc^ then 



. . . , Xn. 



Xi + (-a2)X2 + • • • + (-an)xn = 







which shows that the collection {xi, . . . , Xn} is linearly dependent. I 
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It is important to realize that no linearly independent set of vectors can 
contain the zero vector. To see this, note that if S = {x,, . . . , Xn} and Xi = 0, 
then ax, + 0x2 + • • • + Oxn = for all a E jF, and hence by definition, S is a 
linearly dependent set. 

Theorem 2.3 Let S = {xi, . . . , Xn} C V be a linearly independent set, and let 
S be the linear span of S. Then every v E 5 has a unique representation 

n 

(=1 

where each ai E ^. 

Proof By definition of S, we can always write v = 2a iXi. As to uniqueness, it 
must be shown that if we also have v = SbjXj, then it follows that bj = a i for 
every i = 1, . . . , n. But this is easy since SaiXj = 2biXi implies 2(ai - bi)Xi = 
0, and hence a^ - bi = (since {Xj} is linearly independent). Therefore = hi 
for each i = 1, . . . , n. I 

If S is a finite subset of a vector space V such that V = ^ (the linear span 
of S), then we say that V is finite-dimensional. However, we must define 
what is meant in general by the dimension of V. If S C V is a linearly inde- 
pendent set of vectors with the property that Y = S, then we say that S is a 
basis for V. In other words, a basis for V is a linearly independent set that 
spans V. We shall see that the number of elements in a basis is what is meant 
by the dimension of V. But before we can state this precisely, we must be 
sure that such a number is well-defined. In other words, we must show that 
any basis has the same number of elements. We prove this (see the corollary 
to Theorem 2.6) in several steps. 

Theorem 2.4 Let S be the linear span of S = {xi, . . . , Xn} C V. If k < n and 

{xi, . . . , Xk} is linearly independent, then there exists a linearly independent 
subset of S of the form {xi, . . . , Xk, Xij, . . . , Xi„} whose linear span also 
equals S. 

Proof If k = n there is nothing left to prove, so we assume that k < n. Since 
Xi, . . . , Xk are linearly independent, we let Xj (where j > k) be the first vector 
in S that is a linear combination of the preceding Xi, . . . , Xj_i . If no such j 
exists, then take (ii, . . . , ia) = (k + 1, . . . , n). Then the set of n - 1 vectors Xi, 
. . . , Xj-i , Xj+i , . . . , Xn has a linear span that must be contained in S (since 
this set is just a subset of S). However, if v is any vector in S, we can write v = 
2 f =1 ai Xi where Xj is just a linear combination of the first j - 1 vectors. In 
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other words, v is a linear combination of Xi, . . . , Xj_i, xj+i , . . . , Xn and hence 
these n - 1 vectors also span S. 

We now continue this process by picking out the first vector in this set of 
n - 1 vectors that is a linear combination of the preceding vectors. An identi- 
cal argument shows that the linear span of this set of n - 2 vectors must also 
be S. It is clear that we will eventually obtain a set {xi, . . . , Xk, Xij, . . . , Xi^} 
whose linear span is still S, but in which no vector is a linear combination of 
the preceding ones. This means that the set must be linearly independent 
(Theorem 2.2). I 

Corollary 1 If V is a finite-dimensional vector space such that the set S = 
{xi, . . . , Xm} C V spans V, then some subset of S is a basis for V. 

Proof By Theorem 2.4, S contains a linearly independent subset that also 
spans V. But this is precisely the requirement that S contain a basis for V. I 

Corollary 2 Let V be a finite-dimensional vector space and let {x,, . . . , Xn} 
be a basis for V. Then any element v G V has a unique representation of the 
form 

n 

where each ai G ^f. 

Proof Since {x i} is linearly independent and spans V, Theorem 2.3 shows us 
that any v G V may be written in the form v = 2?=i aiXj where each ai G ^ is 
unique (for this particular basis). I 

It is important to realize that Corollary 1 asserts the existence of a finite 
basis in any finite-dimensional vector space, but says nothing about the 
uniqueness of this basis. In fact, there are an infinite number of possible bases 
for any such space. However, by Corollary 2, once a particular basis has been 
chosen, then any vector has a unique expansion in terms of this basis. 

Example 2.7 Returning to the space JF", we see that any (ai, . . . , an) G 
can be written as the linear combination 

ai(l, 0, . . . , 0) + a2(0, 1, 0, . . . , 0) + • • • + a„(0, . . . , 0, 1) . 

This means that the n vectors 
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ei=(l, 0, 0,...,0) 
e^=(0, 1, 0,...,0) 

e„=(0, 0, 0,...,1) 

span jT". They are also linearly independent since 2aiei = (Ej, . . . , En) = if 
and only if Ej = for all i = 1, . . . , n. The set {e i} is extremely useful, and 
will be referred to es the standard basis for / 

This exEmple leEds us to mEke the following generElizEtion. By En 
ordered basis for e finite-dimensionEl spEce V, we meEn e finite sequence of 
vectors thEt is lineErly independent End spEns V. If the sequence Xi, . . . , Xn is 
En ordered bEsis for V, then the set {xi, . . . , Xn} is a basis for V. In other 
words, the set {x,, . . . , Xn} gives rise to n! different ordered bases. Since 
there is usually nothing lost in assuming that a basis is ordered, we shall 
continue to assume that {xi, . . . , Xn} denotes an ordered basis unless 
otherwise noted. 

Given any (ordered) bEsis {xi, . . . , Xn} for V, we know thEt Eny v £ V 
hEs E unique representEtion v = 2'i=i EjXi . We ceU the scElErs Ei, . . . , En the 
coordinates of v relative to the (ordered) basis {x,, . . . , Xn}. In particular, we 
call a i the ith coordinate of v. Moreover, we now proceed to show that these 
coordinates define an isomorphism between V and 

Since a vector space is also an (additive abelian) group, it is reasonable 
that we mEke the following definition. Let V End W be vector spEces over ^. 
We sEy thEt a mEpping (|): V ^ W is e vector space homomorphism (or, es 
we shEll ceU it lEter, a linear transformation) if 

(|)(x + y) = (t)(x) + (t)(y) 

End 

(j)(EX) = E(j)(x) 

for eU X, y £ V End a ^ ^. This Egrees with our previous definition for 
groups, except thEt now we must tEke into Eccount the multiplication by 
scalars. If (]) is injective, then we say that (j) is En isomorphism. End if (j) is 
bijective, that V and W are isomorphic. 

As before, we define the kernel of (j) to be the set 

Ker (|) = {x e V: (|)(x) = e W} . 
If X, y £ Ker (j) and c £ F we have 
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(|)(x + y) = (|)(x) + (|)(y) = 

and 

(t)(cx) = c(t)(x) = cO = . 

This shows that both x + y and cx are in Ker (j), and hence Ker (|) is a subspace 
of V. Note also that if a = and x £ V then 

(|)(0) = (|)(ax) = a(|)(x) = . 

Alternatively, we could also note that 

m = (|)(x + 0) = (|)(x) + (|)(0) 

and hence (|)(0) = 0. Finally, we see that 

= m = (l)(x + (-x)) = (t)(x) + (t)(-x) 

and therefore 

(i)(-x) = -m ■ 

Our next result is essentially the content of Theorem 1.6 and its corollary. 

Theorem 2.5 Let (j): V ^ W be a vector space homomorphism. Then (j) is an 
isomorphism if and only if Ker (j) = {0}. 

Proof If (|) is injective, then the fact that (|)(0) = implies that we must have 
Ker (|) = {0}. Conversely, if Ker (|) = {0} and (|)(x) = (|)(y), then 

= (t)(x)-(t)(y) = (t)(x-y) 

implies that x - y = 0, or x = y. I 

Now let us return to the above notion of an ordered basis. For any finite- 
dimensional vector space V over ^ and any (ordered) basis {xi, . . . , Xn}, we 
define a mapping (j): V -^^^ by 



for each 

n 



(=1 
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Since 

0(2a,.x,. + = (t)(E(ai + 

= (ai, ...,aj + (&i, 
= (p(LaiXi) + (p(LbiXi) 

and 

0(^) = (l)(k'Za-Xf) = ^(2(A;a;)x,) = (fcaj, ^a„) = /:(ai, a„) 
= /:^(v) 

we see that (|) is a vector space homomorphism. Because the coordinates of 
any vector are unique for a fixed basis, we see that this mapping is indeed 
well-defined and one-to-one. (Alternatively, the identity element in the space 
JF" is (0, ... , 0), and the only vector that maps into this is the zero vector in 
V. Hence Ker (j) = {0} and (]) is an isomorphism.) It is clear that (j) is surjective 
since, given any ordered set of scalars a,, . . . , an G ^, we can define the 

vector V = SajXj G V. Therefore we have shown that V and are isomorphic 
for some n, where n is the number of vectors in an ordered basis for V. 

If V has a basis consisting of n elements, is it possible to find another basis 
consisting of m ;t n elements? Intuitively we guess not, for if this were true 

then V would be isomorphic to as well as to which implies that is 

isomorphic to for m n. That this is not possible should be obvious by 

simply considering the projection of a point in IR^ down onto the plane IR^. 

Any point in IR^ is thus the image of an entire vertical line in IR^, and hence 
this projection can not possibly be an isomorphism. Nevertheless, we proceed 
to prove this in detail beginning with our next theorem. 

Theorem 2.6 Let {xi, . . . , Xn} be a basis for V, and let {yi, . . . , ym} be lin- 
early independent vectors in V. Then m < n. 

Proof Since {x,, . . . , Xn} spans V, we may write each yi as a linear combi- 
nation of the Xj. In particular, choosing ym, it follows that the set 

{ym 5 • • • 5 Xn} 

is linearly dependent (Theorem 2.2) and spans V (since the Xk already do so). 
Hence there must be a proper subset {ym, Xi,, . . . , XiJ with r < n - 1 that 
forms a basis for V (Theorem 2.4). Now this set spans V so that ym-i is a lin- 
ear combination of this set, and hence 
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{ym-1 , ym , Xi„ . . . , XiJ 

is linearly dependent and spans V. By Theorem 2.4 again, we can find a set 
{ym-i , ym , Xj, , . . . , XjJ with s < n - 2 that is also a basis for V. Continuing 
our process, we eventually obtain the set 

{yi, • • • , ym, Xa, Xp, . . . } 

which spans V and must contain at least one of the (since yi is not a linear 
combination of the set {y2, . . . , ym} by hypothesis). This set was constructed 
by adding m - 1 vectors yi to the original set of n vectors x,,, and deleting at 
least m - 1 of the x^ along the way. However, we still have at least one of the 
Xk in our set, and hence it follows that m-l<n-lorm<n. I 

Corollary Any two bases for a finite-dimensional vector space must consist 
of the same number of elements. 

Proof Let {xi, . . . , Xn} and {yi, . . . , ym} be bases for V. Since the yi are lin- 
early independent, Theorem 2.6 says that m < n. On the other hand, the Xj are 
linearly independent so that n < m. Therefore we must have n = m. ■ 

We now return to the proof that is isomorphic to if and only if m = 
n. Let us first show that an isomorphism maps a basis to a basis. 

Theorem 2.7 Let (j): V — * W be an isomorphism of finite-dimensional vector 
spaces. Then a set of vectors {())(vi), . . . , (j)(vn)} is linearly dependent in W if 
and only if the set {vi, . . . , Vn} is linearly dependent in V. 

Proof If the set {v,, . . . , Vn} is linearly dependent, then for some set of 
scalars {a,, . . . , an} not all equal to we have 2f=i aiVi = 0. Applying (j) to 
both sides of this equation yields 

= m = (|)(2aiVi) = 2(|)(aivO = 2ai(|)(Vi) . 

But since not all of the ai are 0, this means that {(t)(Vi)} must be linearly depen- 
dent. 

Conversely, if (t)(vi), . . . , (j)(Vn) are linearly dependent, then there exists a 
set of scalars bi, . . . , bn not all such that 2bi(t)(Vi) = 0. But this means 



= 2bi(|)(vO = 2(|)(bivO = (|)(2bivO 
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which implies that SbjVi = (since Ker (|) = {0}). This shows that the set {Vi} 
is linearly dependent. I 

Corollary If (j): V -> W is an isomorphism of finite-dimensional vector 
spaces, then {(j)(Xi)} = {(j)(xi), . . . , (t)(xn)} is a basis for W if and only if {Xj} = 
{xi, . . . , Xn} is a basis for V. 

Proof Since ^ is an isomorphism, for any vector w G W there exists a unique 
V G V such that (t)(v) = w. If {x i} is a basis for V, then v = 2f =i aiXj and 

w = (t)(v) = (j)(2aiXi) = 2ai(j)(Xi) . 

Hence the (|)(Xi) span W, and they are linearly independent by Theorem 2.7. 

On the other hand, if {(|)(Xi)} is a basis for W, then there exist scalars {bi} 
such that for any v G V we have 

(j)(v) = w = 2bi(t)(Xi) = (t)(2biXi) . 

Since (|) is an isomorphism, this implies that v = SbiXj, and hence {Xj} spans 
V. The fact that it is linearly independent follows from Theorem 2.7. This 
shows that {Xj} is a basis for V. I 

Theorem 2.8 iF" is isomorphic to iF™ if and only if n = m. 

Proof If n = m the result is obvious. Now assume that and ^™ are iso- 
morphic. We have seen in Example 2.7 that the standard basis of consists 
of n vectors. Since an isomorphism carries one basis onto another (corollary to 
Theorem 2.7), any space isomorphic to must have a basis consisting of n 
vectors. Hence by the corollary to Theorem 2.6 we must have m = n. ■ 

Corollary If V is a finite-dimensional vector space over J^, then V is iso- 
morphic to for a unique integer n. 

Proof It was shown following Theorem 2.5 that V is isomorphic to for 
some integer n, and Theorem 2.8 shows that n must be unique. I 

The corollary to Theorem 2.6 shows us that the number of elements in any 
basis for a finite-dimensional vector space is fixed. We call this unique num- 
ber n the dimension of V over ^, and we write dim V = n. Our next result 
agrees with our intuition, and is quite useful in proving other theorems. 
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Theorem 2.9 Every subspace W of a finite-dimensional vector space V is 
finite-dimensional, and dim W < dim V. 

Proof We must show that W has a basis, and that this basis contains at most 
n = dim V elements. If W = {0}, then dim W = < n and we are done. If W 
contains some Xi ^ 0, then let Wi C W be the subspace spanned by Xj. If W = 
Wi, then dim W = 1 and we are done. If W ;t Wi, then there exists some X2 £ 

W with X2 ^ Wi, and we let W2 be the subspace spanned by {xi, X2}. Again, 
if W = W2, then dim W = 2. If W W2, then choose some X3 G W with 
X3 ^ W2 and continue this procedure. However, by Theorem 2.6, there can be 
at most n linearly independent vectors in V, and hence dim W < n. ■ 

Note that the zero subspace is spanned by the vector 0, but {0} is not 
linearly independent so it can not form a basis. Therefore the zero subspace is 
defined to have dimension zero. 

Finally, let us show that any set of linearly independent vectors may be ex- 
tended to form a complete basis. 

Theorem 2.10 Let V be finite-dimensional and S = {xi, . . . , Xm} any set of 
m linearly independent vectors in V. Then there exists a set {xm+i, . . . , Xm+r} 
of vectors in V such that {xi, . . . , Xm+r} is a basis for V. 

Proof Since V is finite-dimensional, it has a basis {v,, . . . , Vn}. Then the set 
{xi, . . . , Xm , Vi, . . . , Vn} spans V so, by Theorem 2.4, we can choose a 
subset {xi, . . . , Xm, Vip . . . , ViJ of linearly independent vectors that span V. 
Letting Vj^ = Xm+i , . . . , vi^ = Xm+r proves the theorem. I 

Exercises 

1. Determine whether or not the three vectors Xi = (2,-l,0),X2 = (l,-l, 1) 
and X3 = (0, 2, 3) form a basis for R^. 

2. In each of the following, show that the given set of vectors is linearly 
independent, and decide whether or not it forms a basis for the indicated 
space: 

(a) {(1, l),(l,-l)}inR2. 

(b) {(2,0, 1), (1,2,0), (0,1,0)} in r3. 

(c) {(1, 0, 0, 0), (1, 1, 0, 0), (1, 1, 1, 0), (1, 1, 1, 1)} in r4. 
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3. Extend each of the following sets to a basis for the given space: 

(a) {(1, 1,0), (2, -2,0)} in r3. 

(b) {(1, 0, 0, 0), (1, 0, 1, 0), (1, 0, 0, 1)} in R\ 

(c) {(1, 1, 0, 0), (1, -1, 0, 0), (1, 0, 1, 0)} in r4 

4. Show that the vectors u = (1 + 2i), v = (1, 1 + i) £ C are linearly 
dependent over C, but linearly independent over R. 

5. Find the coordinates of the vector (3, 1, -4) G relative to the basis Xi = 
(1,1, 1), X2 = (0,1,1) and X3 = (0, 0, 1). 

6. Let IR3[x] be the space of all real polynomials of degree < 3. Determine 
whether or not each of the following sets of polynomials is linearly inde- 
pendent: 

(a) {x^ - 3x2 + 5x + 1^ - x2 + 8x + 2, 2x^ - 4x^ + 9x + 5}. 

(b) {x^ + 4x2 _ 2x + 3, x^ + 6x2 - X + 4^ 3,^3 ^ 3^2 _ 3^ ^ 

7. Let V be a finite-dimensional space, and let W be any subspace of V. 
Show that there exists a subspace W of V such that W fl W = {0} and 
V = W + W (see Exercise 1 . 12 for the definition of W + W). 

8. Let (j): V W be a homomorphism of two vector spaces V and W. 

(a) Show that (j) maps any subspace of V onto a subspace of W. 

(b) Let S' be a subspace of W, and define the set S = {x e V: (|)(x) e S'}. 
Show that S is a subspace of V. 

9. Let V be finite-dimensional, and assume that (j): V ^ V is a surjective ho- 
momorphism. Prove that (j) is in fact an isomorphism of V onto V. 

10. Let V have basis Xi, Xj, . . . , Xn, and let Vi, V2, . . . , Vn be any n elements 
in V. Define a mapping (|): V ^ V by 



ft n 



a.V; 



(=1 



where each aj G ^. 

(a) Show that (j) is a surjective homomorphism. 

(b) When is (j) an isomorphism? 
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2.3 DIRECT SUMS 

We now present some useful ways of constructing a new vector space from 
several given spaces. The reader is advised to think carefully about these con- 
cepts, as they will become quite important later in this book. We also repeat 
our earlier remark that all of the vector spaces that we are discussing are con- 
sidered to be defined over the same field f. 

Let A and B be subspaces of a finite-dimensional vector space V. Then we 
may define the sum of A and B to be the set A + B given by 

A + B = {a + b: a e A and b e B} . 

It is important to note that A and B must both be subspaces of the same space 
V, or else the addition of a G A to b G B is not defined. In fact, since A and B 
are subspaces of V, it is easy to show that A + B is also subspace of V. 
Indeed, given any ai + bi and az + bz in A + B and any k G ^ we see that 

(ai + bi) + (a2 + b2) = (ai + az) + (bi + bz) G A + B 

and 

k(ai + bi) = kai + kbi G A + B 

as required. This definition can clearly be extended by induction to any finite 
collection {Aj} of subspaces. 

In addition to the sum of the subspaces A and B, we may define their 
intersection A fl B by 

A n B = {x G V: X G A and X G B} . 

Since A and B are subspaces, we see that for any x, y G A D B we have both 
X + y G A and x + y G B so that x + y G A fl B, and if x G A fl B then kx G A 
and kx G B so that kx G A n B. Since G A fl B, we then see that A fl B is a 
nonempty subspace of V. This can also be extended to any finite collection of 

subspaces of V. 

Our next theorem shows that the dimension of the sum of A and B is just 
the sum of the dimensions of A and B minus the dimension of their intersec- 
tion. 

Theorem 2.11 If A and B are subspaces of a finite-dimensional space V, 
then 

dim(A + B) = dim A + dim B - dim(A n B) . 
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Proof Since A + B and A fl B are subspaces of V, it follows that both A + B 
and A n B are finite-dimensional (Theorem 2.9). We thus let dim A = m, 
dim B = n and dim A H B = r. 

Let {u,, . . . , Ur} be a basis for A H B. By Theorem 2.10 there exists a set 
{vi, . . . , Vm-r} of linearly independent vectors in V such that 

{Ui, . . . , Ur, Vi, . . . , Vm-r} 

is a basis for A. Similarly, we have a basis 

{Ui, . . . , Ur, Wi, . . . , Wn-r} 

for B. It is clear that the set 

{Ui, . . . , Ur, Vi, . . . , Vm-r, Wi, . . . , Wn-r} 

spans A + B since any a + b G A + B (with a G A and b G B) can be written as 
a linear combination of these r + (m-r) + (n-r) = m + n- r vectors. To 
prove that they form a basis for A + B, we need only show that these m + n - r 
vectors are linearly independent. 

Suppose we have sets of scalars {ai}, {bj} and {Ck} such that 

r m-r n-r 

i=i j=i k=i 

Then 

r m-r n-r 

(=1 j=i k=i 
Since the left side of this equation is an element of A while the right side is an 
element of B, their equality implies that they both belong to A D B, and hence 

n-r r 

for some set of scalars {di}. But {u,, . . . , Ur, w,, . . . , Wn-r} forms a basis for 
B and hence they are linearly independent. Therefore, writing the above equa- 
tion as 

r n-r 
(=1 k=l 

implies that 

di = • • • = dr = Ci = • • • = Cn-r = . 

We are now left with 
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r m-r 



But {Ui, . . . , Ur, Vi 



(=1 j=l 

, . . . , Vm-r} is also linearly independent so that 



ai = • • • = ar 



= bi = • • • = b 



m-r 



= . 



This proves that {u,, . . . , Ur, v,, . . . , Vm-r, w,, . . . , Wn-r} is linearly inde- 
pendent as claimed. The proof is completed by simply noting that we have 
shown 

dim(A + B) = m + n- r = dim A + dim B - dim(A fl B) . ■ 

We now consider a particularly important special case of the sum. If A and 
B are subspaces of V such that A H B = {0} and V = A + B, then we say that 

V is the internal direct sum of A and B. A completely equivalent way of 
defining the internal direct sum is given in the following theorem. 

Theorem 2.12 Let A and B be subspaces of a finite-dimensional vector 
space V. Then V is the internal direct sum of A and B if and only if every 

V £ V can be uniquely written in the form v = a + b where a E A and b E B. 

Proof Let us first assume that V is the internal direct sum of A and B. In 
other words, V = A + B and A fl B = {0}. Then by definition, for any v E V 
we have v = a + b for some a E A and b E B. Suppose we also have v = a' + b' 
where a' E A and b' E B. Then a + b = a' + b' so that a - a' = b' - b. But note 
that a - a' E A and b' - b E B, and hence the fact that A Pi B = {0} implies 
that a - a' = b' - b = 0. Therefore a = a' and b = b' so that the expression for v 
is unique. 

Conversely, suppose that every v E V may be written uniquely in the form 

V = a + b with a E A and b E B. This means that V = A + B, and we must still 
show that A n B = {0}. In particular, if v E A Pi B we may write v = v + 
with V E A and E B, or alternatively, we may write v = + v with E A 
and V E B. But we are assuming that the expression for v is unique, and hence 
we must have v = (since the contributions from A and B must be the same in 
both cases). Thus A fl B = {0} and the sum is direct. I 

We emphasize that the internal direct sum is defined for two subspaces A 
and B of a given space V. As we stated above, this is because the addition of 
two vectors from distinct spaces is not defined. In spite of this, we now pro- 
ceed to show that it is nevertheless possible to define the sum of two distinct 
vector spaces. 
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Let A and B be distinct vector spaces (over the same field ^, of course). 
While the sum of a vector in A and a vector in B makes no sense, we may 
relate these two spaces by considering the Cartesian product A x B defined as 
(see Section 0.1) 

AxB = {(a,b): aGAandbGB} . 

Using the ordered pairs (a, b), it is now easy to turn AxB into a vector space 
by making the following definitions (see Example 2.1). 

First, we say that two elements (a, b) and (a', b') of A x B are equal if and 
only if a = a' and b = b'. Next, we define addition and scalar multiplication in 
the obvious manner by 

(a, b) + (a', b') = (a + a', b + b') 

and 

k(a,b) = (ka,kb) . 

We leave it as an exercise for the reader to show that with these definitions, 
the set A X B defines a vector space V over J^. This vector space is called the 
external direct sum of the spaces A and B, and is denoted by A © B. 

While the external direct sum was defined for arbitrary spaces A and B, 
there is no reason why this definition can not be applied to two subspaces of a 
larger space V. We now show that in such a case, the internal and external 
direct sums are isomorphic. 

Theorem 2.13 If V is the internal direct sum of A and B, then V is isomor- 
phic to the external direct sum A © B. 

Proof If V is the internal direct sum of A and B, then any v G V may be writ- 
ten uniquely in the form v = a + b. This uniqueness allows us to define the 
mapping (j): V ^ A © B by 

(|)(v) = (|)(a + b) = (a, b) . 

Since for any v = a + b and v' = a' + b', and for any scalar k we have 

(|)(v + v') = (a + a', b + b') = (a, b) + (a', b') = (|)(v) + (|)(v') 

and 

(|)(kv) = (ka,kb) = k(a,b) = k(|)(v) 

it follows that (j) is a vector space homomorphism. It is clear that ^ is surjec- 
tive, since for any (a, b) G A © B we have (j)(v) = (a, b) where v = a + b G V. 
Finally, if (|)(v) = (0, 0) then we must have a = b = = v and hence Ker ^ - 
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{0}. This shows that (|) is also injective (Theorem 2.5). In other words, we 
have shown that V is isomorphic to A © B. I 

Because of this theorem, we shall henceforth refer only to the direct sum 
of A and B, and denote this sum by A ® B. It follows trivially from Theorem 
2.11 that 

dim(A©B) = dim A + dim B . 

Example 2.8 Consider the ordinary Euclidean three- space V = R'^. Note that 
any v £ IR'^ may be written as 

(Vi, V2, Vs) = (Vi, V2, 0) + (0, 0, V3) 

which is just the sum of a vector in the xy-plane and a vector on the z-axis. It 
should also be clear that the only vector in the intersection of the xy-plane 
with the z-axis is the zero vector. In other words, defining the space A to be 

the xy-plane and the space B to be the z-axis R\ we see that V = A © B or 

r3 = r2 ® ri. 

On the other hand, if we try to write R^ as the direct sum of the xy-plane A 
with say, the yz-plane B, then the intersection condition is violated since 
A n B is just the entire y-axis. In this case, any vector lying on the y-axis can 
be specified in terms of its components in either the xy-plane or in the yz- 
plane. / 

In many of our later applications we shall need to take the direct sum of 
several vector spaces. While it should be obvious that this follows simply by 
induction from the above case, we go through the details nevertheless. We say 
that a vector space V is the direct sum of the subspaces Wi, . . . , Wr if the 
following properties are true: 

(a) Wi {0} for each i = 1, . . . , r; 

(b) W, n (Wi + • • • + Wi-i + Wi+i + • • • + Wr) = {0} for i = 1, . . . , r; 

(c) V = Wi + ---+Wr. 

If V is the direct sum of the Wi, then we write V = Wi© • • • ©Wr. The gener- 
alization of Theorem 2.12 is the following. 

Theorem 2.14 If Wi, . . . , Wr are subspaces of V, then 

V = Wi© • • • ©Wr 

if and only if every v £ V has a unique representation of the form 
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V = Vi + • • • + Vr 

where Vi £ Wi for each i = 1, . . . , r. 

Proof First assume that V is the direct sum of Wi, . . . , Wr. Given any v G V, 
part (c) in the definition of direct sum tells us that we have 

V = Vi + • • • + Vr 

where Vi E Wi for each i = 1, . . . , r. If we also have another representation 
with v'i e Wi, then 



V = v'l + • • • + V'r 



Vi + -- -+Vr = Vi + -- -+ Vr 

SO that for any i = 1, . . . , r we have 

v'i - Vi = (Vi - v'l) + • • • + (Vi_i - V'i_l) + (Vi+1 - V'i+i) 

+ • • • + (Vr - v'r). 

Since v'i - Vi G Wi and the right hand side of this equation is an element of 
Wi + • • • + Wj-i + Wi+i + • • • + Wr , we see that part (b) of the definition 
requires that v'i - Vi = 0, and hence v'i = Vi. This proves the uniqueness of the 
representation. 

Conversely, assume that each v G V has a unique representation of the 
form V = Vi + • • • + Vr where Vi G Wi for each i = 1, . . . , r. Since part (c) of 
the definition of direct sum is automatically satisfied, we must show that part 
(b) is also satisfied. Suppose 



Since 



we must also have 



VI G Wi n (W2 + • • • + Wr) 
VI G W2 + • • • + Wr 

V, = V2 + • • • + Vr 



for some V2 G W2, . . . , Vr G Wr . But then 

= -Vi + V2 + • • • + Vr 

= + - • • +0 



and 
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are two representations of the vector 0, and hence the uniqueness of the repre- 
sentations implies that Vi = for each i = 1, . . . , r. In particular, the case i = 1 
means that 

Wi n (W2 + • • • + Wr) = {0} . 

A similar argument applies to Wj fl (W2 + • • • + Wi-i + Wi+i + • • • + Wr) for 
any i = 1, . . . , r. This proves part (b) in the definition of direct sum. I 

If V = Wi © • • • © Wr , then it seems reasonable that we should be able to 
form a basis for V by adding up the bases of the subspaces Wi . This is indeed 
the case as we now show. 

Theorem 2.15 Let Wi, . . . , Wr be subspaces of V, and for each i = 1, . . . , r 
let Wi have basis Bi = {Wii, . . . , Winj}. Then V is the direct sum of the Wi if 
and only if the union of bases 

B = U[=i Bi = {Wii, . . . , Wini , . . . , Wrl , . . . , WrnJ 

is a basis for V. 

Proof Suppose that B is a basis for V. Then for any v £ V we may write 
V = (aiiWii + ••• + ai„Wi„^ ) + ••• + (a,iw,i + ••• + a^w^) 

= Wj +••• + Wj. 

where 

w- = a-,w-, H 1- a- w- GW 

1 il il inj inj 1 

and ai j £ y. Now let 

V = w'l + • • • + w'r 

be any other expansion of v, where each w'i £ Wi. Using the fact that Bi is a 
basis for Wi we have 

w'i = b iiWii + • • • + b iniWin; 

for some set of scalars bij . This means that we may also write 

V = (biiWii + • • • + bini Wini) + • • • + (brlWrl + • • • + br Wr nr) • 

However, since B is a basis for V, we may equate the coefficients of Wij in 
these two expressions for v to obtain aij = bij for all i, j. We have thus proved 
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that the representation of v is unique, and hence Theorem 2.14 tells us that V 
is the direct sum of the Wj. 

Now suppose that V is the direct sum of the Wi. This means that any v G 
V may be expressed in the unique form v = Wi + • • • + Wr where Wi G Wi for 
each I = 1, . . . , r. Given that = {wi i , . . . , wi n;} is a basis for Wi, we must 
show that B = UBj is a basis for V. We first note that each Wi G Wi may be 
expanded in terms of the members of Bi , and therefore UBi clearly spans V. It 
remains to show that the elements of B are linearly independent. We first 
write 

(CiiWii + • • • + CiniWim) + • • • + (C rl Wri + • • • + C rnr Wrnr) =0 

and note that 

CiiWii + • • • + CiniWinj G Wi . 

Using the fact that + • • • + = (where each G Wi) along with the unique- 
ness of the representation in any direct sum, we see that for each i = 1, . . . , r 
we must have 

Cii Wii + • • • + Cin; Win; = . 

However, since Bi is a basis for Wi, this means that Cjj = for every i and j, 
and hence the elements of B = UBi are linearly independent. I 

Corollary If V = Wi © • • • © Wr , then 

r 

dim V = 2) dim Wj. 

i= 1 

Proof Obvious from Theorem 2.15. This also follows by induction from 
Theorem 2.11. I 

Exercises 

1. Let W| and Wj be subspaces of R'^ defined by Wi = {(x, y, z): x = y = z} 
and W2 = {(x, y, z): x = 0}. Show that = Wi © W2. 

2. Let Wi be any subspace of a finite-dimensional space V. Prove that there 
exists a subspace W2 of V such that V = Wi © W2. 

3. Let Wi, W2 and W3 be subspaces of a vector space V. Show that 



(Wi n W2) + (Wi n W3) c Wi n (W2 + W3) 
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Give an example in V = IR^ for which equality does not hold. 

4. Let V = F[IR] be as in Exercise 2.1.6. Let W+ and W- be the subsets of V 
defined by W+ = {f G V: f(-x) = f(x)} and W- = {f G V: f(-x) = -f(x)}. 
In other words, W+ is the subset of all even functions, and W- is the 
subset of all odd functions. 

(a) Show that W+ and W- are subspaces of V. 

(b) Show that V = W+eW-. 

5. Let W, and W2 be subspaces of a vector space V. 

(a) Show that W, C W, + W2 and W2 C W, + W2. 

(b) Prove that Wi + W2 is the smallest subspace of V that contains both 
Wi and W2. In other words, if S(Wi, W2) denotes the linear span of Wi 
and W2, show that Wi + W2 = 5(Wi, W2). [Hint: Show that Wi + W2 C 
5(Wi, W2) and5(Wi, W2) C Wi + W2.] 

6. Let V be a finite-dimensional vector space. For any x G V, we define ^x = 
{ax: a G ^ }. Prove that {xi, X2, . . . , Xn} is a basis for V if and only if V = 

7. If A and B are vector spaces, show that A + B is the span of A U B. 
2.4 INNER PRODUCT SPACES 

Before proceeding with the general theory of inner products, let us briefly 
review what the reader should already know from more elementary courses. It 

is assumed that the reader is familiar with vectors in IR^, and we show that for 
any a, b G the scalar product (also called the "dot product") a • b may 
be written as either 

3 

a»b= Jaibi 

i= 1 

where {aj and {bj} are the coordinates of a and b relative to the standard 
basis for IR-^ (see Example 2.7), or as 

a»b = llall llbllcose 

where 6 = Z(a, b ) and 
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i= 1 

with a similar equation for || b || . The symbol || is just the vector space gener- 
alization of the absolute value of numbers, and will be defined carefully below 
(see Example 2.9). For now, just think of || a || as meaning the length of the 
vector a in IR-^. 

Just for fun, for the sake of completeness, and to show exactly what these 
equations depend on, we prove this as a series of simple lemmas. Our first 
lemma is known as the Pythagorean theorem. 

Lemma 2.1 Given a right triangle with sides a, b, and c as shown. 




b 



we have c = a + b . 

Proof Draw the line PQ perpendicular to the hypotenuse c = AB. Note that 
we can now write c as the sum of the two parts Ci and Cj. First observe that the 
triangle ABP is similar to triangle APQ because they are both right triangles 
and they have the angle at A in common (so they must have their third angle 
the same). If we let this third angle be 6 =/-(ABP), then we also have 6 
= l.(APQ). 




Note that the three triangles ABP, APQ and PBQ are all similar, and hence we 
have (remember c = Ci + Cj) 

c, b ^ Ct a 
— = - and — = - . 

be a c 
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Therefore 



C = Cj + = 



from which the lemma follows immediately. I 

Our next lemma is known as the law of cosines. This law, together with 
Lemma 2.1, shows that for any triangle T with sides a < b < c, it is true that 

a^ + = c^ if and only if T is a right triangle. 
Lemma 2.2 For any triangle as shown. 




we have c^ = a^ + b^ - 2ab cos 6. 



Proof Draw a perpendicular to side b as shown: 




By the Pythagorean theorem we have 



= + {h -acosOy 

= (a sin 0)- +(b-a cos 6)^ 

= a} sin^ d + h^ - labcosd + cos^ 6 

= a^ - labcosd 



where we used sin 6 + cos 6=1 which follows directly from Lemma 2.1 



with a = c(sin 9) and b = c(cos 6). I 



We now define the scalar product a • b for any a, b £ IR by 



'b=2)aibi=b' 



i= 1 



where a = (ai, aj, as) and b = (bi, hj, bs). It is easy to see that 
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' (b + c) = 2 aj(bj + Ci) = ^ (ajbj + ajCj) = a»b + a»c 



i= 1 



i= 1 



and similarly, it is easy to show that 



and 



(a + b)»c =a»c + b»c 



(ka) • b = k(a • b) 



where k G R. From the figure below, we see that the Pythagorean theorem 
also shows us that 

3 

II a IF = 2 ajaj = a • a . 



i= 1 




This is the justification for writing || a || to mean the length of the vector a G 

Noting that any two vectors (with a common origin) in IR^ lie in a plane, 
we have the following well-known formula for the dot product. 



Lemma 2.3 For any a, b G IR we have 



a • b = ab cos 6 



where a = || a ||, b = || b || and = Z(a, b ). 



Proof Draw the vectors a and b along with their difference c = a - b: 
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b 



By the law of cosines we have = a^ + - 2ab cos 6, while on the other 
hand 

c^ = ||a-b||2 = (a-b)»(a-b) = a2 + b2-2a»b . 

Therefore we see that a • b = ab cos 6. I 

The main reason that we went through all of this is to motivate the gen- 
eralization to arbitrary vector spaces. For example, if u, v £ R°, then to say 
that 

n 

U»V= ^UiV; 

i= 1 

makes sense, whereas to say that u»v = ||u||||v|| cos 6 leaves one wondering 
just what the "angle" means in higher dimensions. In fact, this will be used 
to define the angle 6. 

We now proceed to define a general scalar (or inner) product (u, v) of vec- 
tors u, V E V. Throughout this section, we let V be a vector space over either 
the real field IR or the complex field C. By way of motivation, we will want 
the inner product { , > applied to a single vector v E V to yield the length (or 

norm) of v, so that ||v|p = {v, v>. But ||v|| must be a real number even if the 
field we are working with is C. Noting that for any complex number z E C we 

have |zp = zz*, we are led to make the following definition. 

Let V be a vector space over ^ (where ^ is either IR or C). By an inner 
product on V (sometimes called the Hermitian inner product), we mean a 
mapping ( , >: V x V ^ ^ such that for all u, v, w E V and a, b G we have 

(IPl) {au + bv, w> = a*{u, w> + b*{v, w>; 
(IP2) (u, V) = (V, u)*; 

(IPS) (u, u> > and (u, u> = if and only if u = 0. 

Using these properties, we also see that 

(u, av + b\v) = (av + bw, u) * 

= (a*(v, u) + b* (w, m)) * 
= a(u, v'j + blu, w) 
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and hence, for the sake of reference, we call this 

(IPT) (u, av + bw) = a(u, v) + b(u, w). 

(The reader should be aware that instead of {au, v> = a*{u, v>, many authors 
define {au, v> = a{u, v> and {u, av) = a*{u, v>. This is particularly true in math- 
ematics texts, whereas we have chosen the convention used by most physics 
texts. Of course, this has no effect on any of our results.) 

A space V together with an inner product is called an inner product 
space. If V is an inner product space over the field C, then V is called a com- 
plex inner product space, and if the field is IR, then V is called a real inner 
product space. A complex inner product space is frequently called a unitary 
space, and a real inner product space is frequently called a Euclidean space. 
Note that in the case of a real space, the complex conjugates in (IPl) and (IP2) 
are superfluous. 

By (IP2) we have (u, u) E IR so that we may define the length (or norm) 
of u to be the nonnegative real number 

Hull = {u, uf^ . 

If ||u|| = 1, then u is said to be a unit vector. If ||v|| j!^ 0, then we can normalize 
V by setting u = v/||v||. One sometimes writes v to mean the unit vector in the 
direction of v, i.e., v = ||v|| v. 

Example 2.9 Let X = (xi, . . . , Xn) and Y = (yi, . . . , yn) be vectors in C°. We 
define 

and leave it to the reader to show that this satisfies (IPl) - (IP3). In the case of 
the space IR°, we have (X, Y) = X»Y = 2xiyi. This inner product is called the 
standard inner product in C" (or IR"). 
We also see that if X, Y e R" then 

\\X-Yf={X-Y,X-Y) = 2(^,-y,y. 

Thus ||X - Y|| is indeed just the distance between the points X = (xi, . . . , Xn) 
and Y = (y,, . . . , yn) that we would expect by applying the Pythagorean theo- 
rem to points in IR". In particular, ||X|| is simply the length of the vector X. / 
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It is now easy to see why we defined the inner product as we did. For 
example, consider simply the space C^. Then with respect to the standard 
inner product on C^, the vector X = (1, j, 0) will have norm ||X|p = (X, X> = 
1 + 1+0 = 2, while if we had used the expression corresponding to the stan- 
dard inner product on IR^, we would have found ||X|p = 1-1+0 = even 
though XjtO, 

Example 2.10 Let V be the vector space of continuous complex- valued 
functions defined on the real interval [a, b]. We may define an inner product 
on V by 

(/, 8) = jy*(x)g(x)dx 

for all f, g £ V. It should be obvious that this satisfies the three required prop- 
erties of an inner product. / 

We now prove the generalization of Theorem 0.7, an important result 
known as the Cauchy-Schwartz inequality. 

Theorem 2.16 Let V be an inner product space. Then for any u, v £ V we 
have 

l(u, v>| < Hull IMI . 

Proof If either u or v is zero the theorem is trivially true. We therefore 
assume that u^O and y ^0. Then, for any real number c, we have (using (IP2) 
and the fact that |zp = zz*) 

II , . ||2 

S V-C^M, vju\\ 

= (v-c(^U, V^M, V-c(m, v)m^ 

= ^v, -c(u, v)(v, - c(m, v) * (m, v) + (m, v) v)(a, m) 

= ||v||^ - 2c|^M, v)| +C-^|^M, v)| ||m||^. 

Now let c = l/||u|p to obtain 

< llvlP - |(u, v>p/||u|p 

or 

l(u, v)|2 < IMP IMP . 

Taking the square root proves the theorem. I 
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Theorem 2.17 The norm in an inner product space V has the following 
properties for all u, v G V and k G jT: 

(Nl) Hull > and ||u|| = if and only if u = 0. 

(N2) llkull = |k| Hull. 

(N3) ||u + v|| < Hull + ||v||. 

Proof Since ||u|| = (u, uf^, (Nl) follows from (IPS). Next, we see that 

IMP = (ku,ku> = IkPllulP 

and hence taking the square root yields (N2). Finally, using Theorem 2.16 and 
the fact that z + z* = 2 Re z < 2|z| for any z G C, we have 

||m + v||^ ={u + v, u + v^ 

= {u, m) + (u, + ^v, + ^v, v) 

= ||m||^ +iu, v) + (m, v)*+||v||^ 
<. ||m||^ + 2|^m, v)| + ||v||^ 
< + 2||m||||v|| + ||v||^ 

-(IHKIMir- 

Taking the square root yields (N3). I 

We note that property (N3) is frequently called the triangle inequality 
because in two or three dimensions, it simply says that the sum of two sides of 
a triangle is greater than the third. Furthermore, we remark that properties 
(Nl) - (N3) may be used to define a normed vector space. In other words, a 
normed vector space is defined to be a vector space V together with a map- 
ping II II : V IR that obeys properties (Nl) - (N3). While a normed space V 
does not in general have an inner product defined on it, the existence of an 
inner product leads in a natural way (i.e., by Theorem 2.17) to the existence of 
a norm on V. 

Example 2.11 Let us prove a simple but useful result dealing with the norm 
in any normed space V. From the properties of the norm, we see that for any 
u, V G V we have 

||m|| = ||m - V + v|| <. ||m - v|| + ||v|| 

and 

||v|| = ||v - M + m|| <. ||m - v|| + ||m||. 
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Rearranging each of these yields 

- ||v|| :S ||m - v|| 

and 

||v|| - ||m|| <. ||m - v||. 

This shows that 

I ||m|| - ||v|| I ^ ||m - v||. / 

Example 2.12 Consider the space V of Example 2.10 and the associated 
inner product {f, g>. Applying Theorem 2.16 we have 

Jj*(x)gix)dx <}^Jjf(x)\ dxj dx^ 

and applying Theorem 2.17 we see that 

The reader might try and prove either of these directly from the definition of 
the integral if he or she wants to gain an appreciation of the power of the 
axiomatic approach to inner products. / 

Finally, let us finish our generalization of Lemmas 2.1 - 2.3. If we repeat 
the proof of Lemma 2.3 using the inner product and norm notations, we find 
that for any u, v G R-^ we have (u, v) = ||u|| ||v|| cos 6. Now let V be any real 
vector space. We define the angle 6 between two nonzero vectors u, v E V by 

(u, v) 

COS0 = f^. 

IrlllMI 

Note that |cos 6| < 1 by Theorem 2.16 so that this definition makes sense. We 
say that u is orthogonal (or perpendicular) to v if {u, v> = 0. If u and v are 

orthogonal, we often write this as u J. v. From the basic properties of the inner 
product, it then follows that (v, u) = (u, v)* = 0* = so that v is orthogonal to 
u also. Thus u J. v if and only if cos 6 = 0. While cos 9 is only defined in a 
real vector space, our definition of orthogonality is valid in any space V over 
J. 
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Exercises 

1. Let X = (xi, X2) and y = (yi, y2) be vectors in IR , and define the mapping 
( , >: IR'^ ^ R by (x, y) = Xiyi - x{y2 - Xjji + 3x2y2. Show that this defines 
an inner product on R^. 

2. Let X = (3, 4) G R^, and evaluate ||x|| with respect to the norm induced by: 

(a) The standard inner product on R^. 

(b) The inner product defined in the previous exercise. 

3. Let V be an inner product space, and let x, y G V. 

(a) Prove the parallelogram law: 

l|x + yiP + ||x - yiP = 2||x|p + 2||y|p . 

(The geometric meaning of this equation is that the sum of the squares of 
the diagonals of a parallelogram is equal to the sum of the squares of the 
sides.) 

(b) Prove the Pythagorean theorem: 

l|x + y|P = ||x|P + ||y|P ifxly . 

4. Find a unit vector orthogonal to the vectors x = (1, 1,2) and y = (0, 1,3) 
inR^. 

5. Let u = (zi, Z2) and v = (wi, W2) be vectors in C^, and define the mapping 
{,>:C2^Rby 

(u, v> = ZiWi* + (1 + i)z{W2* + (1 - OZ2W1* + 3Z2W2* . 
Show that this defines an inner product on C . 

6. Let u = (1 - 2i, 2 + 3i) G and evaluate ||u|| with respect to the norm 
induced by: 

(a) The standard norm on C^. 

(b) The inner product defined in the previous exercise. 

7. Let V be an inner product space. Verify the following polar form identi- 
ties: 

(a) If V is a real space and x, y G V, then 
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{x,y> = (l/4)(||x + y||2-||x-y|p) . 

(b) If V is a complex space and x, y £ V, then 

(X, y> = (l/4)( ||x + yiP - ||x - y|p) + (//4)( ||/ x + y|p - ||/x - y|p) 

(If we were using instead the inner product defined by (ax, y) = a(x, y), 
then the last two terms in this equation would read ||x + jy||.) 

8. Let V = C[0, 1] be the space of continuous real- valued functions defined 
on the interval [0, 1]. Define an inner product on C[0, 1] by 

(a) Verify that this does indeed define an inner product on V. 

(b) Evaluate ||f|| where f = t^ - 2t + 3 e V. 

9. Given a vector space V, we define a mapping d: V x V IR by d(x, y) = 
||x - yll for all x, y e V. Show that: 

(a) d(x, y) > and d(x, y) = if and only if x = y. 

(b) d(x,y) = d(y,x). 

(c) d(x, z) < d(x, y) + d(y, z) (triangle inequality). 

The number d(x, y) is called the distance from x to y, and the mapping d 
is called a metric on V. Any arbitrary set S on which we have defined a 
function d: S x S ^ R satisfying these three properties is called a metric 
space. 

10. Let {ci, . . . , Cn} be an orthonormal basis for a complex space V, and let 
X £ V be arbitrary. Show that 

n 

(a) ^ = 2e,.(e,., x) . 

Nr=ii(^.'^>r- 

11. Show equality holds in the Cauchy-Schwartz inequality if and only if one 
vector is proportional to the other. 



2.5 ORTHOGONAL SETS 

If a vector space V is equipped with an inner product, then we may define a 
subspace of V that will turn out to be extremely useful in a wide variety of 
applications. Let W be any subset of such a vector space V. (Note that W need 
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not be a subspace of V.) We define the orthogonal compliment of W to be 

the set W"^ given by 

W-^ = {v E V: (v, w) = for all w E W} . 

Theorem 2.18 Let W be any subset of a vector space V. Then W"^ is a sub- 
space of V. 

Proof We first note that E W"^ since for any v E V we have 

(0, v> = (Ov, v) = 0(v, v> = . 

To finish the proof, we simply note that for any u, v E W"^, for any scalars a, 
b E ^, and for every w E W we have 

(au + bv, w> = a*(u, w) + b*(v, w) = a*0 + b*0 = 

so that au + bv E W"^. I 

Consider the space IR^ with the usual Cartesian coordinate system (x, y, z). 
If we let W = IR^ be the xy-plane, then W"^ = IR^ is just the z-axis since the 
standard inner product on IR^ shows that any v E IR^ of the form (0, 0, c) is 
orthogonal to any w E IR^ of the form (a, b, 0). Thus, in this case anyway, we 

see that W © W-^ = R^. We will shortly prove that W © W-^ = V for any 
inner product space V and subspace W C V. Before we can do this however, 
we must first discuss orthonormal sets of vectors. 

A set {Vi} of nonzero vectors in a space V is said to be an orthogonal set 
(or to be mutually orthogonal) if {Vj, Vj) = for i 9^ j. If in addition, each Vj is 
a unit vector, then the set {Vi} is said to be an orthonormal set and we write 

{Vi, Vj > = 6ij 

where the very useful symbol 6ij (called the Kronecker delta) is defined by 

' [0 if i^j' 

Theorem 2.19 Any orthonormal set of vectors {Vj} is linearly independent. 

Proof If 2aiVi = for some set of scalars {ai}, then 
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= (Vj, 0) = {Vj, 2aiVi> = 2i ai{Vj, V;) = 2i ai8ij = a,- 

so that Oj = for each j, and hence {Vi} is linearly independent. I 

Note that in the proof of Theorem 2.19 it was not really necessary that 
each Vi be a unit vector. Any orthogonal set would work just as well. 

Theorem 2.20 If {vi, Vj, . . . , Vn} is an orthonormal set in V and if w £ V is 
arbitrary, then the vector 

u = w - 2i (Vi, w>Vi 

is orthogonal to each of the Vi. 

Proof We simply compute {vj, u>: 

{v^.,m) = (v^.,w-2,(v,, w)v,) 

= (v^,>v)-2,{v,,w){v^.,v,) 
= (v^.,w)-2,(v,,w)6,^ 

= {^.'^)-^;'^) = • ■ 

The numbers Ci = (Vj, w> are frequently called the Fourier coefficients of 
w with respect to Vj. In fact, we leave it as an exercise for the reader to show 
that the expression ||w - Si aiViH achieves its minimum precisely when Ej = Ci 
(see Exercise 2.5.4). Furthermore, we also leave it to the reader (see Exercise 
2.5.5) to show that 

n 

Vl |2 II ||2 

(=1 

which is called Bessel's inequality. 

As we remarked earlier, most mathematics texts write {u, av> = a*{u, v> 
rather than {u, av> = a(u, v>. In this case. Theorem 2.20 would be changed to 
read that the vector 

U = W - 2i (W, Vi>Vi 

is orthogonal to each Vj. 

Example 2.13 The simplest and best known example of an orthonormal set 
is the set {ei} of standard basis vectors in IR°. Thus 
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and clearly 



e,=(l, 0, 0,...,0) 
e^=(0, 1, 0,...,0) 

e„=(0, 0, 0,...,1) 
(ei, Cj) = Ci • Cj = 6ij 



since for any X = (xi, . . . , Xn) and Y = (yi, . . . , yn) in R", we have 

(x,y) = x.7 = 2;x,j, . 

(It would perhaps be better to write the unit vectors as ei rather than ei, but this 
will generally not cause any confusion.) / 

Example 2.14 Let V be the space of continuous complex-valued functions 
defined on the real interval [-n, Jt]. As in Example 2. 10, we define 



(/' S) = f_J*{x)gix)dx 



for all f, g E V. We show that the set of functions 



2nj 



1/2 



for n = 1, 2, . . . forms an orthonormal set. 
If m = n, then 



fn) = {U fn) = ^O-'-^'- dx = i-£ = 1 



If m ;t n, then we have 



In 

\ gi(n-m)x 

2jt i(n - m) 
sin(n - m)ji 
7c{n - m) 



= 
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since sin njt = for any integer n. (Note that we also used the fact that 

smd = —(e'^ -e-'^) 

ir ' 

which follows from the Euler formula mentioned in Chapter 0.) Therefore, 
(fm, fn> = Smn- That the set {fn} is orthonormal is of great use in the theory of 
Fourier series. / 

We now wish to show that every finite-dimensional vector space with an 
inner product has an orthonormal basis. The proof is based on the famous 
Gram-Schmidt orthogonalization process, the precise statement of which we 
present as a corollary following the proof. 

Theorem 2.21 Let V be a finite-dimensional inner product space. Then there 
exists an orthonormal set of vectors that forms a basis for V. 

Proof Let dim V = n and let {ui, . . . , Un} be a basis for V. We will construct 
a new basis {wi, . . . , Wn} such that {Wj, Wj) = 8ij. To begin, we choose 

>Vi = Mj / ||Mi|| 

(w^, Wj) = ^Mj / ||Mi||, Mj /||Wi||) = (1 / "l) 

II l|2 II ||2 

(l/pj )||m,|| = 1 

and hence Wi is a unit vector. We now take U2 and subtract off its "projection" 
along Wi. This will leave us with a new vector V2 that is orthogonal to Wi. 
Thus, we define 

V2 = U2 - (Wi, U2>Wi 

so that 

{Wi, V2> = (Wi, U2> - {Wi, U2XW1, Wi> = 

(this also follows from Theorem 2.20). If we let 

W2 = V2/IIV2II 

then {wi, W2} is an orthonormal set (that V2 ^ will be shown below). 

We now go to U3 and subtract off its projection along Wi and W2. In other 
words, we define 

V3 = U3 - {W2, U3>W2 - (Wi, U3>Wi 



SO that 
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so that (wi, V3> = {w2, V3> = 0. Choosing 

W3 = V3/IIV3II 

we now have an orthonormal set {wi, W2, W3}. 

It is now clear that given an orthonormal set {wi, . . . , w^}, we let 

k 

!- = l 

SO that Vk+i is orthogonal to Wi, . . . , w^ (Theorem 2.20), and hence we define 

Wk+l = Vk+l/||Vk+l|| . 

It should now be obvious that we can construct an orthonormal set of n 
vectors from our original basis of n vectors. To finish the proof, we need only 
show that Wi, . . . , Wn are linearly independent. 

To see this, note first that since Ui and U2 are linearly independent, Wi and 
U2 must also be linearly independent, and hence V2 ^ by definition of linear 
independence. Thus W2 exists and {wi, W2} is linearly independent by 
Theorem 2.19. Next, {w,, W2, U3} is linearly independent since Wj and W2 are 
in the linear span of u, and U2. Hence \-i that W3 exists, and Theorem 

2.19 again shows that {wi, W2, W3} is linearly independent. 

In general then, if {wi, . . . , Wk} is linearly independent, it follows that 
{wi, . . . , Wk, Uk+i} is also independent since {wi, . . . , Wk} is in the linear 
span of {Ui, . . . , Uk}. Hence Vk+i ^ and Wk+i exists. Then {wi, . . . , Wk+i} is 
linearly independent by Theorem 2.19. Thus {Wi, . . . , Wn} forms a basis for 
V, and (Wi, Wj> = 6ij. I 

Corollary (Gram-Schmidt process) Let {ui, . . . , Un} be a hnearly indepen- 
dent set of vectors in an inner product space V. Then there exists a set of 
orthonormal vectors Wi, . . . , Wn ^ V such that the linear span of {ui, . . . , Uk} 
is equal to the linear span of {wi, . . . , Wk} for each k = 1, . . . , n. 

Proof This corollary follows by a careful inspection of the proof of Theorem 
2.21. I 

We emphasize that the Gram-Schmidt algorithm (the "orthogonalization 
process" of Theorem 2.21) as such applies to any inner product space, and is 
not restricted to only finite-dimensional spaces (see Chapter 12). 

We are now ready to prove our earlier assertion. Note that here we require 
W to be a subspace of V. 
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Theorem 2.22 Let W be a subspace of a finite-dimensional inner product 
space V. Then V = W © W-^. 

Proof By Theorem 2.9, W is finite-dimensional. Therefore, if we choose a 
basis {vi, . . . , Vk} for W, it may be extended to a basis {vi, . . . , Vn} for V 
(Theorem 2.10). Applying Theorem 2.21 to this basis, we construct a new 
orthonormal basis {ui, . . . , Un} for V where 

r 

for r = 1, . . . , n and some coefficients arj (determined by the Gram-Schmidt 
process). In particular, we see that Ui, . . . , Uk are all in W, and hence they 
form an orthonormal basis for W. 

Since {u,, . . . , Un} are orthonormal, it follows that u^+i , . . . , Un are in 
W"^ (since (Ui, Uj) = for all i < k and any j = k + 1, . . . , n). Therefore, given 
any x £ V we have 

X = aiUi + • • • + anUn 

where 

aiUi + • • • + akUk £ W 

and 

ak+iUk+i + • • • + anUn e W-^ . 

This means that V = W + W-^, and we must still show that W n W-^ = {0}. 
But if y £ W n W"^, then {y, y> = since y £ W"^ implies that y is orthogonal 
to any vector in W, and in particular, y £ W. Hence y = by (IP3), and it 
therefore follows that W n W-^ = {0}. I 

Corollary If V is finite-dimensional and W is a subspace of V, then (W"^)"^ = 
W. 

Proof Given any w E W we have (w, v) = for all v G W"^. This implies that 
w e (W-^)-^ and hence W C (W-^)-^. By Theorem 2.22, V = W © W-^ and 
hence 

dimV = dimW + dimW-^ 

(Theorem 2. 1 1). But W-^ is also a subspace of V, and hence V = W-^ © (W-^)-^ 
(Theorem 2.22) which implies 

dim V = dim W-^ + dim(W-^)-^ . 

Therefore, comparing these last two equations shows that dim W = 
dim(W-^)-^. This result together with W C (W-^)-^ implies that W = (W-^)-^. I 
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Finally, note that if {Ci} is an orthonormal basis for V, then any x £ V 
may be written as x = 2i Xjej where 

(ej, x> = (ej, 2i xjei) = 2i Xi(ej, d) = 2; Xi6ij = Xj . 

Therefore we may write 

X = 2i {e;, x>ei 
which is a very useful expression. 

Example 2.15 Consider the following basis vectors for IR-^: 

ui = (3,0,4) U2 = (-l,0,7) U3 = (2,9, 11). 

Let us apply the Gram-Schmidt process (with the standard inner product on 
R^) to obtain a new orthonormal basis for IR-^. Since ||ui|| = V9+16 = 5, we 
define 

Wi = Ui/5 = (3/5,0,4/5) . 
Next, using {wi, Uj) = -3/5 + 28/5 = 5 we let 

V, = (-1,0, 7) -(3, 0,4) = (-4,0,3) . 
Since HvzH = 5, we have 

W2 = (-4/5, 0, 3/5) . 
Finally, using {wi, us) = 10 and (wj, us) = 5 we let 

V3 = (2, 9, 1 1) - (-4, 0, 3) - (6, 0, 8) = (0, 9, 0) 
and hence, since ||v3|| = 9, our third basis vector becomes 

W3 = (0,1,0) . 

We leave it to the reader to show that {wi, W2, W3} does indeed form an 
orthonormal basis for IR^. / 

We will have much more to say about inner product spaces after we have 
treated linear transformations in detail. For the rest of this book, unless explic- 
itly stated otherwise, all vector spaces will be assumed to be finite- 
dimensional. In addition, the specific scalar field ^ will generally not be 
mentioned, but it is to be understood that all scalars are elements of ^. 
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Exercises 

1. Let W be a subset of a vector space V. Prove the following: 

(a) 0-^ = V and V"^ = 0. 

(b) w n w-^ = {0}. 

(c) Wi C W2 implies W2-^ C Wi"^. 

2. Let U and W be subspaces of a finite-dimensional inner product space V. 
Prove the following: 

(a) (U + w)-^ = u-^ n w-^. 

(b) (u n w)-^ = m + w-^. 

3. Let {ci, . . . , Cn} be an orthonormal basis for an arbitrary inner product 
space V. If u = 2i UiCi and v = 2i VjCj are any vectors in V, show that 

n 

{u, v) = 2)«;*V,- 

(=1 

(this is just the generalization of Example 2.9). 

4. Suppose {ei, . . . , en} is an orthonormal set in a vector space V, and x is 
any element of V. Show that the expression 

n 
k=l 

achieves its minimum value when each of the scalars a^ is equal to the 
Fourier coefficient c^ = (Ck, x). [Hint: Using Theorem 2.20 and the 
Pythagorean theorem (see Exercise 2.4.3), add and subtract the term 
2g=i CkCk in the above expression to conclude that 





2 




n 




n 


^-^Ck^k 


< 


k=l 



2 



for any set of scalars ak.] 

5. Let {ci, . . . , Cn} be an orthonormal set in an inner product space V, and 
let Ck = {ek, x> be the Fourier coefficient of x £ V with respect to ek. 
Prove Bessel's inequality: 
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[Hint: Use the definition of the norm along with the obvious fact that < 
||x-2e=iCkek|p.] 

6. Find an orthonormal basis (relative to the standard inner product) for the 
following subspaces: 

(a) The subspace W of spanned by the vectors Ui = (1, i, 0) and U2 = 
(1, 2, 1 - i). 

(b) The subspace W of R"^ spanned by Ui = (1, 1, 0, 0), U2 = (0, 1, 1, 0) 
and U3 = (0,0, 1, 1). 

7. Consider the space IR'^ with the standard inner product. 

(a) Convert the vectors Ui = (1, 0, 1), U2 = (1, 0, -1) and U3 = (0, 3, 4) to 

an orthonormal basis {ei, e2, es} of IR'^. 

(b) Write the components of an arbitrary vector x = (xi, X2, X3) £ IR^ in 
terms of the basis {ei}. 

8. Let V = C[0, 1] be the inner product space defined in Exercise 2.4.8. Find 
an orthonormal basis for V generated by the functions {1, x, x^, x^}. 

9. Let V and W be isomorphic inner product spaces under the vector space 
homomorphism (|): V ^ W, and assume that (|) has the additional property 
that 

||(t)(xO-(t)(X2)|| = Ik-X2|| . 

Such a (|) is called an isometry, and V and W are said to be isometric 
spaces. (We also note that the norm on the left side of this equation is in 

W, while the norm on the right side is in V. We shall rarely distinguish 
between norms in different spaces unless there is some possible ambigu- 
ity.) Let V have orthonormal basis {vi, . . . , Vn} so that any x G V may 
be written as x = SxjVj. Prove that the mapping (j): V — * defined by 
(t)(x) = (xi, . . . , Xn) is an isometry of V onto IR" (with the standard inner 
product). 

10. Let {ei, e2, es} be an orthonormal basis for IR^, and let {ui, U2, U3} be 

three mutually orthonormal vectors in IR^. Let ux^ denote the ith comp- 
onent of ux with respect to the basis {ei}. Prove the completeness 
relation 

3 
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11. Let W be a finite-dimensional subspace of a possibly infinite-dimensional 
inner product space V. Prove that V = W ® W"^. [Hint: Let {wi, . . . , Wk} 

be an orthonormal basis for W, and for any x E V define 

k 

(=1 

and X2 = X - xi. Show that xi + X2 G W + W"^, and that W n W"^ = {0}.] 



CHAPTER 3 



Linear Equations and 
Matrices 



In this chapter we introduce matrices via the theory of simultaneous linear 
equations. This method has the advantage of leading in a natural way to the 
concept of the reduced row-echelon form of a matrix. In addition, we will for- 
mulate some of the basic results dealing with the existence and uniqueness of 
systems of linear equations. In Chapter 5 we will arrive at the same matrix 
algebra from the viewpoint of linear transformations. 



3.1 SYSTEMS OF LINEAR EQUATIONS 

Let sli, . . . , an, y be elements of a field !F, and let Xi, . . . , Xn be unknowns 
(also called variables or indeterminates). Then an equation of the form 

ai Xi + • • • + an Xn = y 

is called a linear equation in n unknowns (over The scalars a^ are called 
the coefficients of the unknowns, and y is called the constant term of the 
equation. A vector (ci, . . . , Cn) ^ is called a solution vector of this equa- 
tion if and only if 

ai ci + • • • + an Cn = y 
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in which case we say that (ci, . . . , Cn) satisfies the equation. The set of all 
such solutions is called the solution set (or the general solution). 

Now consider the following system of m linear equations in n 
unknowns: 

We abbreviate this system by 

n 
7=1 

If we let Si denote the solution set of the equation 2j a^Xj = yi for each i, then 
the solution set S of the system is given by the intersection S = flSi. In other 
words, if (ci, . . . , Cn) ^ is a solution of the system of equations, then it is a 
solution of each of the m equations in the system. 

Example 3.1 Consider this system of two equations in three unknowns over 
the real field R: 

The vector (3, 1, 3) £ IR^ is not a solution of this system because 

2(3) - 3(1) + 3 = 6 

while 

3 + 5(1) - 2(3) = 1 * \1 . 
However, the vector (5, 1, -1) £ IR^ is a solution since 

2(5) - 3(1) + (-1) = 6 

and 

5 + 5(l)-2(-l) = 12 . / 

Associated with a system of linear equations are two rectangular arrays of 
elements of ^ that turn out to be of great theoretical as well as practical 
significance. For the system 2j a^Xj = y^, we define the matrix of coefficients 
A as the array 
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A = 



a 



11 



a 



12 



a 



m2 



a 



In 



*2n 



a 



mnl 



and the augmented matrix as the array aug A given by 



aug A = 



*21 



n2 



*22 



a 



In 



«2„ y2 



Wml ^m2 '^mn yn) 



In general, we will use the term matrix to denote any array such as the 
array A shown above. This matrix has m rows and n columns, and hence is 
referred to as an m x n matrix, or a matrix of size m x n. By convention, an 
element ajj G ^ of A is labeled with the first index referring to the row and the 
second index referring to the column. The scalar ajj is usually called the i, ']th 
entry (or element) of the matrix A. We will frequently denote the matrix A 
by the symbol (ay). 

Another rather general way to define a matrix is as a mapping from a sub- 
set of all ordered pairs of positive integers into the field J^. In other words, we 
define the mapping A by A(i, j) = aij for every 1 < i < m and 1 < j < n. In this 
sense, a matrix is actually a mapping, and the m x n array written above is just 
a representation of this mapping. 

Before proceeding with the general theory, let us give a specific example 
demonstrating how to solve a system of linear equations. 



Example 3.2 Let us attempt to solve the following system of linear equa- 
tions: 



That our approach is valid in general will be proved in our first theorem 
below. 

Multiply the first equation by 1/2 to get the coefficient of Xi equal to 1: 
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JCj + (1 / 2)^2 - X3 = -3 / 2 

Multiply the first equation by -1 and add it to the second to obtain a new sec- 
ond equation, then multiply the first by -4 and add it to the third to obtain a 
new third equation: 

(1 / 2)x2 - = -3 / 2 
-(7/ 2)^2 +2x3 = 19/2 

Multiply the second by -2/7 to get the coefficient of X2 equal to 1, then mul- 
tiply this new second equation by 3 and add to the third: 

Xj + (1 / 2)^2 - X3 = -3 / 2 
a;2-(4/7)x3 = -19/7 
(2 / 7)X3 = 6/7 

Multiply the third by 7/2, then add 4/7 times this new equation to the second: 

Xj + (1 / 2)^2 - X3 = -3 / 2 
X2 = -1 
X3 = 3 



Add the third equation to the first, then add -1/2 times the second equation to 
the new first to obtain 

JCi = 2 

^2 =-1 

X3 = 3 

This is now a solution of our system of equations. While this system could 
have been solved in a more direct manner, we wanted to illustrate the system- 
atic approach that will be needed below. / 

Two systems of linear equations are said to be equivalent if they have 
equal solution sets. That each successive system of equations in Example 3.2 
is indeed equivalent to the previous system is guaranteed by the following 
theorem. 



Theorem 3.1 The system of two equations in n unknowns over a field !F 
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a^^Xi+ai2X2+--- + a^„x„=b^ 



with an ;t is equivalent to the system 



fljjXj + 0^2X2 H I- <^l„Xn - bi 
^22-^2 ^2n^n ~ ^2 



in which 



for each i = 1, . . . , n and 



a'2i = 321 aii - an a2i 



b'2 = a2i bi - an b2 



Proof Let us define 

n 
7=1 

SO that (1) may be written as the system 



1^2 =h 



(1) 



(2) 



(!') 



(2') 



while (2) is just 

^21^1 ~ ^11^2 ~ ^21^1 ~ ^11^2 
If (xi, . . . , Xn) E is a solution of (1'), then the two equations 

and hence also 

^21 Li - L2 = a2i bi - an b2 

are all true equations. Therefore every solution of (1') also satisfies (2'). 

Conversely, suppose that we have a solution (xi, . . . , Xn) to the system 
(2'). Then clearly 

^21 Ll = ^21 bi 



is a true equation. Hence, subtracting the second of (2') from this gives us 
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^21 Li - (a2i Li - Ell L2) — &21 bi - (a2i bi - an b2) 

or 

an L2 = an b2 . 

Thus L2 = bz is also a true equation. This shows that any solution of (2') is a 
solution of (1') also. I 

We point out that in the proof of Theorem 3.1 (as well as in Example 3.2), 
it was only the coefficients themselves that were of any direct use to us. The 
unknowns Xj were never actually used in any of the manipulations. This is the 
reason that we defined the matrix of coefficients (ay). What we now proceed 
to do is to generalize the above method of solving systems of equations in a 
manner that utilizes this matrix explicitly. 

Exercises 

1. For each of the following systems of equations, find a solution if it exists: 

(a) x+2y-3z = -l (b) 2x+ y-2z = l0 

3x- y + 2z= 7 3x + 2y + 2z= I 

5jc + 33;-4z= 2 5x + 4y+3z= 4 

(c) X + 2y - 3z = 6 
2x- y + 4z= 2 
4x + 3y-2z= 14 

2. Determine whether or not the each of the following two systems is equiva- 
lent (over the complex field): 

(a) x-y = and 3x + >> = 
2x+y=0 x+y=0 

(b) -x+ y+ 4z = and x- z = 

x + 3y+ Sz = y + 3z = 

(l/2);c+ 3; + (5/2)z = 

(c) 2x + (-l + i)y+ t = 

3y - 2iz + 5t = Q 



and 
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(l + i7 2)jc+ Sy-iz- t = 
i2/3)x-il/2)y + z + lt = 

3.2 ELEMENTARY ROW OPERATIONS 

The important point to realize in Example 3.2 is that we solved a system of 
linear equations by performing some combination of the following operations: 

(a) Change the order in which the equations are written. 

(b) Multiply each term in an equation by a nonzero scalar. 

(c) Multiply one equation by a nonzero scalar and then add this new 
equation to another equation in the system. 

Note that (a) was not used in Example 3.2, but it would have been necessary if 
the coefficient of Xi in the first equation had been 0. The reason for this is that 
we want the equations put into echelon form as defined below. 

We now see how to use the matrix aug A as a tool in solving a system of 
linear equations. In particular, we define the following so-called elementary 
row operations (or transformations) as applied to the augmented matrix: 

(a) Interchange two rows. 

(P) Multiply one row by a nonzero scalar. 

(y) Add a scalar multiple of one row to another. 

It should be clear that operations (a) and ((3) have no effect on the solution set 
of the system and, in view of Theorem 3.1, that operation (y) also has no 



The next two examples show what happens both in the case where there is 
no solution to a system of linear equations, and in the case of an infinite 
number of solutions. In performing these operations on a matrix, we will let Ri 
denote the ith row. We leave it to the reader to repeat Example 3.2 using this 
notation. 

Example 3.3 Consider this system of linear equations over the field R: 



effect. 



x+3y + 2z 
2x+ y- z 
-x + 2y + 3z 



4 



5 



7 



The augmented matrix is 
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/ 1 3 2 7^ 
2 1-15 
-12 3 4 



and the reduction proceeds as follows. We first perform the following elemen- 
tary row operations: 

/I 3 2 7^ 
i?2 - 2i?i ^ -5 -5 -9 
/?3 + /?i^lo 5 5 11, 



Now, using this matrix, we obtain 





(1 


3 


2 


7\ 







5 


5 


9 


i?3 + i?2 ^ 


.0 








2. 



It is clear that the equation Oz = 2 has no solution, and hence this system has 
no solution. / 

Example 3.4 Let us solve the following system over the field R: 

3Xj + 2X2 ~ ^3"^ 2-^4 = 

-2xi + 5x2 - 3-^3 - 3x4 = 26 
We have the matrix aug A given by 





-2 


2 


-1 


-14^ 


3 


2 


-1 


2 


17 


2 


3 


-1 


-1 


18 


-2 


5 


-3 


-3 


26^ 



and hence we obtain the sequence 











-2 


2 


-1 


-14\ 


R, 


-3i?i 







8 


-7 


5 


59 


R3 


-2/?i 







7 


-5 


1 


46 


Ra 


+ 2i?i 




.0 


1 


1 


-5 


-2. 
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1 


-2 


2 


-1 


1 /I \ 
-14 







1 


1 


-5 


-2 










-15 


45 


75 


R,-1R,^ 







-12 


36 


60^ 






^1 


-2 


2 


-1 


-14^ 









1 


1 


-5 


-2 


(-l/15)i?3^ 








1 


-3 


-5 


(-l/12)/?4^ 


.0 





1 


-3 


-5. 



We see that the third and fourth equations are identical, and hence we have 
three equations in four unknowns: 

It is now apparent that there are an infinite number of solutions because, if 
we let c E IR be any real number, then our solution set is given by X4 = c, X3 = 
3c - 5, X2 = 2c + 3 and Xi = -c + 2. / 

Two m X n matrices are said to be row equivalent if one can be trans- 
formed into the other by a finite number of elementary row operations. We 
leave it to the reader to show that this defines an equivalence relation on the 
set of all m X n matrices (see Exercise 3.2. 1). 

Our next theorem is nothing more than a formalization of an earlier 
remark. 

Theorem 3.2 Let A and B be the augmented matrices of two systems of m 
linear equations in n unknowns. If A is row equivalent to B, then both systems 
have the same solution set. 

Proof If A is row equivalent to B, then we can go from the system repre- 
sented by A to the system represented by B by a succession of the operations 
(a), (b) and (c) described above. It is clear that operations (a) and (b) have no 
effect on the solutions, and the method of Theorem 3.1 shows that operation 
(c) also has no effect. I 

A matrix is said to be in row-echelon form if successive rows of the 
matrix start out (from the left) with more and more zeros. In particular, a 



= -14 
= -2 
= -5 
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matrix is said to be in reduced row-echelon form if it has the following 
properties (which are more difficult to state precisely than they are to under- 
stand): 

(1) All zero rows (if any) occur below all nonzero rows. 

(2) The first nonzero entry (reading from the left) in each row is equal to 
1. 

(3) If the first nonzero entry in the ith row is in the jith column, then 
every other entry in the jith column is 0. 

(4) If the first nonzero entry in the ith row is in the jith column, then ji < 
j2< • • • . 

We will call the first (or leading) nonzero entries in each row of a row- 
echelon matrix the distinguished elements of the matrix. Thus, a matrix is in 
reduced row-echelon form if the distinguished elements are each equal to 1, 
and they are the only nonzero entries in their respective columns. 

Example 3.5 The matrix 



(1 


2 


-3 





1 \ 








5 


2 


-4 











7 


3 


.0 











0. 



is in row-echelon form but not in reduced row-echelon form. However, the 
matrix 



/I 





5 





2\ 





1 


2 





4 











1 


7 


.0 











0. 



is in reduced row-echelon form. Note that the distinguished elements of the 
first matrix are the numbers 1, 5 and 7, and the distinguished elements of the 
second matrix are the numbers 1, 1 and 1. / 

The algorithm detailed in the proof of our next theorem introduces a tech- 
nique generally known as Gaussian elimination. 

Theorem 3.3 Every m x n matrix A is row equivalent to a reduced row- 
echelon matrix. 
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Proof This is essentially obvious from Example 3.4. The detailed description 
which follows is an algorithm for determining the reduced row-echelon form 
of a matrix. 

Suppose that we first put A into the form where the leading entry in each 
nonzero row is equal to 1, and where every other entry in the column contain- 
ing this first nonzero entry is equal to 0. (This is called simply the row- 
reduced form of A.) If this can be done, then all that remains is to perform a 
finite number of row interchanges to achieve the final desired reduced row- 
echelon form. 

To obtain the row-reduced form we proceed as follows. First consider row 
1. If every entry in row 1 is equal to 0, then we do nothing with this row. If 
row 1 is nonzero, then let ji be the smallest positive integer for which ay, ^ 
and multiply row 1 by (aijj)"'. Next, for each i ;t 1 we add -aijj times row 1 to 
row i. This leaves us with the leading entry a^j of row 1 equal to 1, and every 
other entry in the jV/z column equal to 0. 

Now consider row 2 of the matrix we are left with. Again, if row 2 is equal 
to there is nothing to do. If row 2 is nonzero, assume that the first nonzero 
entry occurs in column (where ^ ji by the results of the previous para- 
graph). Multiply row 2 by (ajjj)"' so that the leading entry in row 2 is equal to 
1, and then add -aijj times row 2 to row i for each i ^ 2. Note that these opera- 
tions have no effect on either column ji, or on columns 1, . . . , ji of row 1. 

It should now be clear that we can continue this process a finite number of 
times to achieve the final row-reduced form. We leave it to the reader to take 
an arbitrary matrix (ay) and apply successive elementary row transformations 
to achieve the desired final form. I 

While we have shown that every matrix is row equivalent to at least one 
reduced row-echelon matrix, it is not obvious that this equivalence is unique. 
However, we shall show in the next section that this reduced row-echelon 
matrix is in fact unique. Because of this, the reduced row-echelon form of a 
matrix is often called the row canonical form. 

Exercises 

1. Show that row equivalence defines an equivalence relation on the set of all 
matrices. 

2. For each of the following matrices, first reduce to row-echelon form, and 
then to row canonical form: 
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(I 


-2 


3 


-1^ 




(I 


2 


-1 


2 1\ 


(a) 


2 


-1 


2 


2 


(b) 


2 


4 


1 


-2 3 




,3 


1 


2 


3y 






6 


2 


-6 5^ 




(I 


3 


-1 


2^ 















1 


-5 


3 










(c) 


















2 


-5 


3 


1 












^4 


1 


1 


5. 











3. For each of the following systems, find a solution or show that no solution 
exists: 



(a) 


x+ y+ z=l 


(b) 


x-y+2z=\ 




2x-3y + lz = 




x+ y+ z = 2 




3x-2y + Sz = 4- 




2x - y+ z = 5 


(c) 


x-y + 2z = 4 


(d) 


x+3y+ z = 2 




3x + y + 4z = 6 




2x + ly + 4z = 6 




x+y+ z=l 




x+ y-4z=l 


(e) 


x+ 3y+ z = 


if) 


2x- 3; + 5z = 19 




2x + ly + 4z = 




x + 5>'-3z = 4 




x+ y-4z=0 




3x + 2y + 4z = 5 


(8) 


2x- y + 5z = 19 








x + 5y-3z = 4 








3x + 2y + 4z = 25 







4. Let fi, f2 and fs be elements of F[IR] (i.e., the space of all real-valued func- 
tions defined on IR). 

(a) Given a set {xi, Xj, X3} of real numbers, define the 3x3 matrix F(x) = 
(fi(Xj)) where the rows are labelled by i and the columns are labelled by j. 
Prove that the set {fj is linearly independent if the rows of the matrix F(x) 
are linearly independent. 

(b) Now assume that each fj has first and second derivatives defined on 
some interval (a, b) C IR, and let fi^^ denote the jth derivative of fj (where 
fi(^> is just fi). Define the matrix W(x) = (f^^-^ >(x)) where 1 < i, j < 3. 
Prove that {fj} is linearly independent if the rows of W(x) are independent 
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for some x £ (a, b). (The determinant of W(x) is called the Wronskian of 
the set of functions {fi}.) 

Show that each of the following sets of functions is linearly independent: 

(c) fi(x) = -X^ + X + 1, f2(x) = x^ + 2x, f3(x) = x^ - 1. 

(d) fi(x) = exp(-x), f2(x) = X, f3(x) = exp(2x). 

(e) fi(x) = exp(x), f2(x) = sin x, f3(x) = cos x. 



5. Let 



(3 -1 



A = 



2 1 
1 -3 



2\ 

1 





Determine the values of Y = (yi, y2, ys) for which the system SiaijXj = yi 
has a solution. 

6. Repeat the previous problem with the matrix 



A = 



Z' 3 -6 2 -l\ 
-2413 
11 
1-210 



3.3 ROW AND COLUMN SPACES 

We will find it extremely useful to consider the rows and columns of an arbi- 
trary m X n matrix as vectors in their own right. In particular, the rows of A 
are to be viewed as vector n- tuples Ai, . . . , Am where each Aj = (au, . . . , 

ain) ^ Similarly, the columns of A are to be viewed as vector m-tuples 

A\ . . . , A° where each A^ = (ay, . . . , amj) ^ For notational clarity, we 

should write AJ as the column vector 



but it is typographically easier to write this horizontally whenever possible. 
Note that we label the row vectors of A by subscripts, and the columns of A 
by superscripts. 
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Since each row Aj is an element of the set of all rows of a matrix can 
be used to generate a new vector space V over J^. In other words, V is the 
space spanned by the rows Aj, and hence any v £ V may be written as 

m 

(=1 

where each Cj £ J^. The space V (which is apparently a subspace of is 
called the row space of A. The dimension of V is called the row rank of A, 
and will be denoted by rr(A). Since V is a subspace of and dim = n, it 
follows that rr(A) = dim V < n. On the other hand, V is spanned by the m 
vectors Aj, so that we must have dim V < m. It then follows that rr(A) < 
min{m, n}. 

In an exactly analogous manner, we define the column space W of a 
matrix A as that subspace of spanned by the n column vectors AJ. Thus 
any w £ W is given by 

n 

>i 

The column rank of A, denoted by cr(A), is given by cr(A) = dim W and, as 
above, we must have cr(A) < min{m, n}. 

An obvious question is whether a sequence of elementary row operations 
changes either the row space or the column space of a matrix. A moments 
thought should convince you that the row space should not change, but it may 
not be clear exactly what happens to the column space. These questions are 
answered in our next theorem. While the following proof appears to be rather 
long, it is actually quite simple to understand. 

Theorem 3.4 Let A and A be row equivalent m x n matrices. Then the row 
space of A is equal to the row space of A, and hence rr(A) = rr(A). 
Furthermore, we also have cr(A) = cr(A). (However, note that the column 
space of A is not necessarily the same as the column space of A.) 

Proof Let V be the row space of A and V the row space of A. Since A and A 
are row equivalent, A may be obtained from A by applying successive ele- 
mentary row operations. But then each row of A is a linear combination of 
rows of A, and hence V C V. On the other hand, A may be obtained from A 
in a similar manner so that V C V. Therefore V = V and hence rr(A) = rr(A). 

Now let W be the column space of A and W the column space of A. 
Under elementary row operations, it will not be true in general that W = W, 
but we will show it is still always true that dim W = dim W. Let us define the 
mapping f: W W by 
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^ n \ n 

I i=l 

In other words, if we are given any linear combination of the columns of A, 
then we look at the same linear combination of columns of A. In order to 

show that this is well-defined, we must show that if 2aiA' = 2biAS then 
f(2aiA^) = f(2biA^). This equivalent to showing that if 2ciA^ = then 
f(2ciAi) = because if 2(3; - bi)A^ = and f(2(ai - bi)A^) = 0, then 

= / (2 (a, - Z., ) A' ) = 2 ( a, - Z7, ) A' = 2fl, A^ - 2&, A' 
= /(2a,A')-/(2^,,A') 



and therefore 
Now note that 



f(2aiAi) = f(2biAi) 



and 



/(2(Z7,A' + c,A')) = /(2(Z7, + c,) A') = 2(&, + c,) A' 

= 2/?. A' + 2c,. A' = /(2Z7,.A') + / (2c,.A') 

f(k(2ciAi)) = f(2(kCi)Ai) = 2(kCi) A' = MQlcA^) 



so that f is a vector space homomorphism. If we can show that W and W are 
isomorphic, then we will have cr(A) = dim W = dim W = cr(A). Since f is 
clearly surjective, we need only show that Ker f = {0} for each of the three 
elementary row transformations. 

In the calculations to follow, it must be remembered that 



and 



Ai — (ail, • • • 5 Sin) 
/a.\ 



A' = 



Since 



2ciAi — (2ciaii, . . . , 2ciain) 



we see that 2ciAi = if and only if 2ciaij = for every j = 1, 
Similarly, 

2ciA^ = (2ciaii, . . . , 2cianii) 



so that 2ciA' = if and only if 2ciaji = for every j = 1, . . . , m (remember 
that we usually write a column vector in the form of a row vector). 
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We first consider a transformation of type a. For definiteness, we inter- 
change rows 1 and 2, although it will be obvious that any pair of rows will 
work. In other words, we define Ai = Aj, Aj = Ai and Aj = Aj for j = 3, . . . , 
n. Therefore 

f(2ciA0 = SciA' = (2cia2i, 2ciaii, Scjasi, . . . , Sciami) • 

If 

SciA^ = 

then 

2ciaii = 

for every j = 1, . . . , m and hence we see that f(2ciA^) = 0. This shows that f is 
well-defined for type a transformations. Conversely, if 

f(2ciA') = 

then we see that again 

Scia,; = 

for every j = 1, . . . , m since each component in the expression 2ciA ' = 

must equal 0. Hence 2ciA^ = if and only if f(2ciA^) = 0, and hence Ker f = 
{0} for type a transformations (which also shows that f is well-defined). 

We leave it to the reader (see Exercise 3.3.1) to show that f is well-defined 
and Ker f = {0} for transformations of type |3, and we go on to those of type y. 
Again for definiteness, we consider the particular transformation Ai = Ai + 
kAj and Aj = Aj for j = 2, . . . , m. Then 

/ C^cA' ) = 2^4' = 2c,. (ai,. + ka2i , a2,- , . . . , ) 

If 

2ciAi = 

then 

Scia,; = 

for every j = 1, . . . , m so that 2ciA' = and f is well-defined for type y 
transformations. Conversely, if 

2ciAi = 

then 

Sciaji = 
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for j = 2, . . . , m, and this then shows that Sciaii = also. Thus 2ciA' = 

implies that 2ciA' = 0, and hence 2ciA* = if and only if f(2ciA') = 0. This 
shows that Ker f = {0} for type y transformations also, and f is well-defined. 

In summary, by constructing an explicit isomorphism in each case, we 
have shown that the column spaces W and W are isomorphic under all three 
types of elementary row operations, and hence it follows that the column 
spaces of row equivalent matrices must have the same dimension. I 

Corollary If A is the row-echelon form of A, then 2ciA' = if and only if 
SciA^ = 0. 

Proof This was shown explicitly in the proof of Theorem 3.4 for each type of 
elementary row operation. I 

In Theorem 3.3 we showed that every matrix is row equivalent to a 
reduced row-echelon matrix, and hence (by Theorem 3.4) any matrix and its 
row canonical form have the same row space. Note though, that if the original 
matrix has more rows than the dimension of its row space, then the rows 
obviously can not all be linearly independent. However, we now show that the 
nonzero rows of the row canonical form are in fact linearly independent, and 
hence form a basis for the row space. 

Theorem 3.5 The nonzero row vectors of an m x n reduced row-echelon 
matrix R form a basis for the row space of R. 

Proof From the four properties of a reduced row-echelon matrix, we see that 
if R has r nonzero rows, then there exist integers ji, . . . , j r with each ji < n 
and ji < • • • < jr such that R has a 1 in the ith row and iith column, and every 
other entry in the iith column is (it may help to refer to Example 3.5 for 
visualization). If we denote these nonzero row vectors by Ri, . . . , Rr then any 
arbitrary vector 

has Cj as its jith coordinate (note that v may have more than r coordinates if r < 
n). Therefore, if v = we must have each coordinate of v equal to 0, and 
hence Cj = for each i = 1, . . . , r. But this means that the Ri are linearly 
independent, and since {Ri} spans the row space by definition, we see that 
they must in fact form a basis. I 
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Corollary If A is any matrix and R is a reduced row-echelon matrix row 
equivalent to A, then the nonzero row vectors of R form a basis for the row 
space of A. 

Proof In Theorem 3.4 we showed that A and R have the same row space. 
The corollary now follows from Theorem 3.5. I 

Example 3.6 Let us determine whether or not the following matrices have 
the same row space: 





^1 


2 


-1 


3\ 


A = 


2 


4 


1 


-2 




.3 


6 


3 


-7. 



B = 



'\ 2 -4 ir 

^2 4 -5 14^ 



We leave it to the reader to show (and you really should do it) that the reduced 
row-echelon form of these matrices is 



A = 



/I 2 1/3^ 
1-8/3 




B = 



/I 2 1/3' 



1-8/3 



Since the the nonzero rows of the reduced row-echelon form of A and B are 
identical, the row spaces must be the same. / 

Now that we have a better understanding of the row space of a matrix, let 
us go back and show that the reduced row-echelon form of a given matrix is 
unique. We first prove a preliminary result dealing with the row-echelon form 
of two matrices having the same row space. 

Theorem 3.6 Let A = (ay) be a row-echelon matrix with distinguished ele- 
ments aij, , a2j2 , . . . , arj, and let B = (by) be another row-echelon matrix with 
distinguished elements biki , b2k2 bsks • Assume that A and B have the 

same row space (and therefore the same number of columns). Then the dis- 
tinguished elements of A are in the same position as those of B, i.e., r = s and 
ji — ki, ^2 — • • • 5 jr = . 

Proof Since A = if and only if B = 0, we need only consider the nontrivial 
case where r > 1 and s > 1. First suppose that ji < kj. This means that the jith 
column of B is equal to (0, ... , 0). Since A and B have the same row space, 
the first row of A is just a linear combination of the rows of B. In particular, 
we then have aijj = 2i Cibijj for some set of scalars Ci (see the proof of 
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Theorem 3.4). But by, = for every i, and hence aijj = which contradicts the 
assumption that aijj is a distinguished element of A (and must be nonzero by 
definition). We are thus forced to conclude that j, > k,. However, we could 
clearly have started with the assumption that k, < j,, in which case we would 
have been led to conclude that ki > ji. This shows that we must actually have 
ji = ki. 

Now let A' and B' be the matrices which result from deleting the first row 
of A and B respectively. If we can show that A' and B' have the same row 
space, then they will also satisfy the hypotheses of the theorem, and our con- 
clusion follows at once by induction. 

Let R = (ai, 3.2, ... , an) be any row of A' (and hence a row of A), and let 
Bi, . . . , Bm be the rows of B. Since A and B have the same row space, we 
again have 

m 
i=l 

for some set of scalars di. Since R is not the first row of A and A' is in row- 
echelon form, it follows that a^ = for i = ji = kj. In addition, the fact that B is 
in row-echelon form means that every entry in the kith column of B must be 
except for the first, i.e., bik, 9^ 0, b2ki = • • • = bmki = 0. But then 

= ak, = di biki + d2 b2ki + • • • + dm bmki = di biki 

which implies that di = since biki ^ 0. This shows that R is actually a linear 
combination of the rows of B', and hence (since R was arbitrary) the row 
space of A' must be a subspace of the row space of B'. This argument can 
clearly be repeated to show that the row space of B' is a subspace of the row 
space of A', and hence we have shown that A' and B' have the same row 
space. I 

Theorem 3.7 Let A = (aij) and B = (by) be reduced row-echelon matrices. 
Then A and B have the same row space if and only if they have the same 
nonzero rows. 

Proof Since it is obvious that A and B have the same row space if they have 
the same nonzero rows, we need only prove the converse. So, suppose that A 
and B have the same row space. Then if Aj is an arbitrary nonzero row of A, 
we may write 

Ai = 2,c,B, (1) 

where the Br are the nonzero rows of B. The proof will be finished if we can 
show that Cr = for r ;t i and Ci = 1. 
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To show that Cj = 1, let ayj be the first nonzero entry in Aj, i.e., ayj is the 
distinguished element of the ith row of A. Looking at the jith component of 
(1) we see that 

= 2,c,b,j^ (2) 

(see the proof of Theorem 3.4). From Theorem 3.6 we know that bij; is the 
distinguished element of the ith row of B, and hence it is the only nonzero 
entry in the jith column of B (by definition of a reduced row-echelon matrix). 
This means that (2) implies ayj = Ci byj . In fact, it must be true that aij; = byj = 
1 since A and B are reduced row-echelon matrices, and therefore Ci = 1. 

Now let bkjk be the first nonzero entry of Bk (where k ^ i). From (1) again 
we have 

^ik^^r^K ■ (3) 

Since B is a reduced row-echelon matrix, bkj,, = 1 is the only nonzero entry in 
the ji,th column of B, and hence (3) shows us that ay^ = Ckbkjk • But from 
Theorem 3.6, akj^ is a distinguished element of A, and hence the fact that A is 
row-reduced means that ay^ = for i ;t k. This forces us to conclude that Ck = 
for k ?£ i as claimed. I 



Suppose that two people are given the same matrix A and asked to trans- 
form it to reduced row-echelon form R. The chances are quite good that they 
will each perform a different sequence of elementary row operations to 
achieve the desired result. Let R and R' be the reduced row-echelon matrices 
that our two students obtain. We claim that R = R'. Indeed, since row equiva- 
lence defines an equivalence relation, we see from Theorem 3.4 that the row 
spaces of R and R' will be the same. Therefore Theorem 3.7 shows us that the 
rows of R must equal the rows of R'. Hence we are justified in calling the 
reduced row-echelon form of a matrix the row canonical form as mentioned 
earlier. 



Exercises 

1. In the proof of Theorem 3.4, show that Ker f = {0} for a type (3 operation. 

2. Determine whether or not the following matrices have the same row 
space: 





fl -2 -1) 






A = 




B = 






.3 -4 5, 




(3 3 -ij 
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3. Show that the subspace of IR'^ spanned by the vectors Ui = (1, 1, -1), = 
(2, 3, -1) and U3 = (3, 1, -5) is the same as the subspace spanned by the 
vectors Vi = (1, -1, -3), V2 = (3, -2, -8) and V3 = (2, 1, -3). 

4. Determine whether or not each of the following sets of vectors is linearly 
independent: 

(a) ui = (1, -2, 1), u, = (2, 1, -1) and U3 = (7, -4, 1). 

(b) ui = (1, 2, -3), U2 = (1, -3, 2) and U3 = (2, -1, 5). 

5. (a) Suppose we are given an m x n matrix A = (ajj), and suppose that one 
of the columns of A, say A\ is a linear combination of the others. Show 
that under any elementary row operation resulting in a new matrix A, the 
column A^ is the same linear combination of the columns of A that A^ is of 
the columns of A. In other words, show that all linear relations between 
columns are preserved by elementary row operations. 

(b) Use this result to give another proof of Theorem 3.4. 

(c) Use this result to give another proof of Theorem 3.7. 

3.4 THE RANK OF A MATRIX 

It is important for the reader to realize that there is nothing special about the 
rows of a matrix. Everything that we have done up to this point in discussing 
elementary row operations could just as easily have been done with columns 
instead. In particular, this means that Theorems 3.4 and 3.5 are equally valid 
for column spaces if we apply our elementary transformations to columns 
instead of rows. This observation leads us to our next fundamental result. 

Theorem 3.8 If A = (aij) is any m x n matrix over a field !f, then rr(A) = 
cr(A). 

Proof Let A be the reduced row-echelon form of A. By Theorem 3.4 it is 
sufficient to show that rr(A) = cr(A). If ji < • • • < jr are the columns containing 

the distinguished elements of A, then {A^', . . . , A^"^} is a basis for the column 
space of A, and hence cr(A) = r. (In fact, these columns are just the first r 
standard basis vectors in JT".) But from the corollary to Theorem 3.5, we see 
that rows Ai, . . . , Ar form a basis for the row space of A, and thus rr(A) = r 
also. I 

In view of this result, we define the rank r(A) of a matrix A as 
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r(A) = rr(A) = cr(A) . 



Our next theorem forms the basis for a practical method of finding the rank of 
a matrix. 



Theorem 3.9 If A is any matrix, then r(A) is equal to the number of nonzero 
rows in the (reduced) row-echelon matrix row equivalent to A. (Alternatively, 
r(A) is the number of nonzero columns in the (reduced) column-echelon 
matrix column equivalent to A.) 

Proof Noting that the number of nonzero rows in the row-echelon form is the 
same as the number of nonzero rows in the reduced row-echelon form, we see 
that this theorem follows directly from the corollary to Theorem 3.5. I 

If A is an n x n matrix such that a^ = for i^] and a^ = 1, then we say that 
A is the identity matrix of size n and write this matrix as In. Since the size is 
usually understood, we will generally simply write I. If I = (Ijj), then another 
useful way of writing this is in terms of the Kronecker delta as 1^ = 8ij. 
Written out, I has the form 



/ = 



/I ••• 0^ 
1 ••• 



^0 ••• 1/ 



Theorem 3.10 If A is an n x n matrix of rank n, then the reduced row- 
echelon matrix row equivalent to A is the identity matrix In. 

Proof This follows from the definition of a reduced row-echelon matrix and 
Theorem 3.9. I 

Example 3.7 Let us find the rank of the matrix A given by 



A = 



( 1 


2 


-3^ 


2 


1 





-2 


-1 


3 


.-1 


4 


-2. 
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To do this, we will apply Theorem 3.9 to columns instead of rows (just for 
variety). Proceeding with the elementary transformations, we obtain the fol- 
lowing sequence of matrices: 



( 1 





°1 




2 


-3 


6 




-2 


3 


-3 




.-1 


6 


-5. 






t 


t 


A2- 


-2A1 


A3 


+ 3A 



( 1 





) 


2 


-1 





-2 


1 


1 


.-1 


2 


7/3^ 



t t 

(1/3)A2 (1/3)(A3 + 2A2) 



1^1 





\ 





1 











1 


.3 


1/3 


7/3^ 




t 


T 



A^ + 2A^ -(A^-A^) 

Thus the reduced column-echelon form of A has three nonzero columns, 
so that r(A) = cr(A) = 3. We leave it to the reader (see Exercise 3.4.1) to show 
that the row canonical form of A is 



(1 





0\ 





1 











1 


.0 





0. 



and hence r(A) = cr(A) = rr(A) as it should. / 
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Exercises 

1. Verify the row-canonical form of the matrix in Example 3.7. 

2. Let A and B be arbitrary m x n matrices. Show that r(A + B) < r(A) + r(B). 

3. Using elementary row operations, find the rank of each of the following 
matrices: 



(«) 



(c) 



/I 


3 


1 


-2 


-3\ 


1 


4 


3 


-1 


-4 


2 


3 


-4 


-7 


-3 


.3 


8 


1 


-7 


-8. 


/ 1 


3^ 











-2 








5 


-1 








.-2 


I 









(b) 



1 


2 


-3^ 


2 


1 





-2 


-1 


3 


-1 


4 


-2. 



^5 -1 n 



(J) 



2 1 -2 
-7 12 



4. Repeat the previous problem using elementary column operations. 



3.5 SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS 

We now apply the results of the previous section to the determination of some 
general characteristics of the solution set to systems of linear equations. We 
will have more to say on this subject after we have discussed determinants in 
the next chapter. 

To begin with, a system of linear equations of the form 

n 

^aijXj=0, i = l,...,m 

is called a homogeneous system of m linear equations in n unknowns. It is 
obvious that choosing Xi = X2 = • • • = Xn = will satisfy this system, but this 
is not a very interesting solution. It is called the trivial (or zero) solution. 
Any other solution, if it exists, is referred to as a nontrivial solution. 
A more general type of system of linear equations is of the form 

n 

2«y^;=yi. «■ = !,..., m 
7=1 

where each yi is a given scalar. This is then called a nonhomogeneous system 
of linear equations. Let us define the column vector 
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Y = (yi,...,yin) e . 

From our discussion in the proof of Theorem 3.4, we see that a^Xj is just Xj 
times the ith component of the jth column AJ £ Thus our system of non- 
homogeneous equations may be written in the form 



7=1 





(a ) 
"11 




(a ) 
"12 






In 




^21 










2n 










+ • 








\^ml) 













= Y 



where this vector equation is to be interpreted in terms of its components. (In 
the next section, we shall see how to write this as a product of matrices.) It 
should also be obvious that a homogeneous system may be written in this 
notation as 

7=1 

Let us now look at some elementary properties of the solution set of a 
homogeneous system of equations. 

Theorem 3.11 The solution set S of a homogeneous system of m equations 
in n unknowns is a subspace of 

Proof Let us write our system as Sja^Xj = 0. We first note that S ^0 since 
(0, . . . , 0) £ is the trivial solution of our system. If u = (ui, . . . , Un) £ 
and V = (vi, . . . , Vn) ^ are both solutions, then 

2jaij(Uj + Vj) = 2jaijUj + 2jaijVj = 

so that u + V G S. Finally, if c G ^ then we also have 

2jaij(cUj) = cSjaijUj = 

so that cu G S. I 



If we look back at Example 3.4, we see that a system of m equations in n > 
m unknowns will necessarily result in a nonunique, and hence nontrivial, solu- 
tion. The formal statement of this fact is contained in our next theorem. 
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Theorem 3.12 Let a homogeneous system of m equations in n unknowns 
have the m x n matrix of coefficients A. Then the system has a nontrivial 
solution if and only if r(A) < n. 

Proof By writing the system in the form SjXjAJ = 0, it is clear that a non- 
trivial solution exists if and only if the n column vectors AJ G ^™ are linearly 
dependent. Since the rank of A is equal to the dimension of its column space, 
we must therefore have r(A) < n. ■ 

It should now be clear that if an n x n (i.e., square) matrix of coefficients 
A (of a homogeneous system) has rank equal to n, then the only solution will 
be the trivial solution since reducing the augmented matrix (which then has 
the last column equal to the zero vector) to reduced row-echelon form will 
result in each variable being set equal to zero (see Theorem 3.10). 

Theorem 3.13 Let a homogeneous system of linear equations in n unknowns 
have a matrix of coefficients A. Then the solution set S of this system is a sub- 
space of with dimension n - r(A). 

Proof Assume that S is a nontrivial solution set, so that by Theorem 3.12 we 
have r(A) < n. Assume also that the unknowns Xi, . . . , Xn have been ordered 
in such a way that the first k = r(A) columns of A span the column space (this 

is guaranteed by Theorem 3.4). Then the remaining columns A'^'^^ . . . , A° 
may be written as 

k 

A' = ^bfjAJ, i = k + l, ... , n 

and where each bjj G y. If we define bii = -1 and by = for j ;t i and j > k, 
then we may write this as 

n 

^hijA' =0, i = k + l, ... , n 

(note the upper limit on this sum differs from the previous equation). Next we 
observe that the solution set S consists of all vectors x G such that 

2^jAJ=0 

,7=1 

and hence in particular, the n - k vectors 



b« = (bu,...,bin) 
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for each i = k + 1, . . . , n must belong to S. We show that they in fact form a 
basis for S, which is then of dimension n - k. 

To see this, we first write out each of the b^^^: 

=(&,,,i,...,Z.,,i„-l, 0, 0,...,0) 
b^'''^=ibk.2v-,h.2k^0^-l 0,...,0) 

d(«)=(Z7„i,...,Z7„„ 0,0,. ..,0,-1) . 
Hence for any set {cj} of n - k scalars we have 

n I n n \ 

i=k+l \i=k+l i=k+l / 

and therefore 

if and only if Ck+i = • • • = Cn = 0. This shows that the b*^^^ are linearly inde- 
pendent. (This should have been obvious from their form shown above.) 
Now suppose that d = (di, . . . , dn) is any solution of 

2^/^=0 . 

7=1 

Since S is a vector space (Theorem 3.11), any linear combination of solutions 
is a solution, and hence the vector 



i=k+l 

must also be a solution. In particular, writing out each component of this 
expression shows that 



yj = + 2 "^'^^ 



and hence the definition of the b^ shows that y = (yi, . . . , yk, 0, . . . , 0) for 
some set of scalars yi. Therefore, we have 

o=23'/'=23'/' 

7 = 1 ,7 = 1 

and since {A\ . . . , A'^} is linearly independent, this implies that yj = for 
each j = 1, . . . , k. Hence y = so that 



It 

J = - 2 dib^^ 



i=k+\ 
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and we see that any solution may be expressed as a linear combination of the 
b(i). 

Since the b*^^^ are linearly independent and we just showed that they span 
S, they must form a basis for S. I 

Suppose that we have a homogeneous system of m equations in n > m 
unknowns, and suppose that the coefficient matrix A is in row-echelon form 
and has rank m. Then each of the m successive equations contains fewer and 
fewer unknowns, and since there are more unknowns than equations, there 
will be n - m = n - r(A) unknowns that do not appear as the first entry in any 
of the rows of A. These n - r(A) unknowns are called free variables. We may 
arbitrarily assign any value we please to the free variables to obtain a solution 
of the system. 

Let the free variables of our system be Xi, , . . . , Xi^ where k = n- m = n- 
r(A), and let Vs be the solution vector obtained by setting Xi^ equal to 1 and 
each of the remaining free variables equal to 0. (This is essentially what was 
done in the proof of Theorem 3.13.) We claim that v,, . . . , v^ are linearly 
independent and hence form a basis for the solution space of the (homoge- 
neous) system (which is of dimension n - r(A) by Theorem 3.13). 

To see this, we basically follow the proof of Theorem 3.13 and let B be 
the k X n matrix whose rows consist of the solution vectors Vs . For each s, our 
construction is such that we have Xi^ = 1 and Xi^ = for r s (and the remain- 
ing m = n - k unknowns are in general nonzero). In other words, the solution 
vector Vs has a 1 in the position of Xj^, while for r s the vector Vr has a in 
this same position. This means that each of the k columns corresponding to 
the free variables in the matrix B contains a single 1 and the rest zeros. We 
now interchange column 1 and column ii, then column 2 and column ij, . . . , 
and finally column k and column ik. This yields the matrix 

0-00 b,,,, - b,: 

0-00 b^,,, - 

1 b,,,, - b^^ 

where the entries bi k+i , . . . , bin are the values of the remaining m unknowns 
in the solution vector Vj. Since the matrix C is in row-echelon form, its rows 
are independent and hence r(C) = k. However, C is column-equivalent to B, 
and therefore r(B) = k also (by Theorem 3.4 applied to columns). But the rows 
of B consist precisely of the k solution vectors Vs, and thus these solution vec- 
tors must be independent as claimed. 



C = 



'\ 
1 
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Example 3.8 Consider the homogeneous system of linear equations 

x + 2y-4z+3w- t = 
x + 2y-2z + 2w+ t = 
2x + 4y-2z + 3w + 4t = 

If we reduce this system to row-echelon form, we obtain 

x + 2y-4z+3w- t = 

(*) 

2z- w + 2t = 

It is obvious that the rank of the matrix of coefficients is 2, and hence the 
dimension of the solution space is 5 - 2 = 3. The free variables are clearly y, 
w and t. The solution vectors Vs are obtained by choosing (y= l,w = 0,t = 0), 
(y = 0, w = 1, t = 0) and (y = 0, w = 0, t = 1). Using each of the these in the 
system (*), we obtain the solutions 

Vi = (-2, 1, 0, 0, 0) 
V2=(-l, 0,1/2, 1, 0) 
V3 = (-3, 0,-1, 0, 1) 

Thus the vectors Vi, Vj and V3 form a basis for the solution space of the homo- 
geneous system. / 

We emphasize that the corollary to Theorem 3.4 shows us that the solution 

set of a homogeneous system of equations is unchanged by elementary row 
operations. It is this fact that allows us to proceed as we did in Example 3.8. 

We now turn our attention to the solutions of a nonhomogeneous system 
of equations SjajjXj = yi . 

Theorem 3.14 Let a nonhomogeneous system of linear equations have 
matrix of coefficients A. Then the system has a solution if and only if r(A) = 
r(aug A). 

Proof Let c = (Ci, . . . , Cn) be a solution of SjajjXj = yi. Then writing this as 

2j CjAJ = Y 

shows us that Y is in the column space of A, and hence 



r(aug A) = cr(aug A) = cr(A) = r(A) 
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Conversely, if cr(aug A) = r(aug A) = r(A) = cr(A), then Y is in the column 
space of A, and hence Y = ScjAJ for some set of scalars Cj . But then the 
vector c = (ci, . . . , Cn) is a solution since it obviously satisfies SjaijXj = yj. I 

Using Theorem 3.13, it is easy to describe the general solution to a non- 
homogeneous system of equations. 

Theorem 3.15 Let 

n 
7=1 

be a system of nonhomogeneous linear equations. If u = (ui, . . . , Un) £ is a 
solution of this system, and if S is the solution space of the associated homo- 
geneous system, then the set 

u + S = {u + v:veS} 

is the solution set of the nonhomogeneous system. 

Proof If w = (Wi, . . . , Wn) ^ is any other solution of 2jaijXj = yi, then 

2jaij(Wj - Uj) = SjaijWj - SjaijUj = y; - y; = 

so that w - u £ S, and hence w = u + v for some v £ S. Conversely, if v £ S 
then 

2jaij(Uj + Vj) = Sj-aijUj + Sjaij-Vj = yj + = yj 
so that u + V is a solution of the nonhomogeneous system. I 
Theorem 3.16 Let A be an n x n matrix of rank n. Then the system 

7=1 

has a unique solution for arbitrary vectors Y E 

Proof Since Y = SA^Xj, we see that Y E is just a linear combination of 
the columns of A. Since r(A) = n, it follows that the columns of A are linearly 
independent and hence form a basis for But then any Y E has a unique 
expansion in terms of this basis (Theorem 2.4, Corollary 2) so that the vector 
X with components Xj must be unique. I 
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Example 3.9 Let us find the complete solution set over the real numbers of 
the nonhomogeneous system 

3xj + X2+ + 4x4 = 1 

Xj + 7x2 ~ ll^'^s + 13x4 = -13 
llxj+ X2 + 12x3 + 10X4 = 9 



We assume that we somehow found a particular solution u = (2, 5, 1, -3) G 
IR"^, and hence we seek the solution set S of the associated homogeneous sys- 
tem. The matrix of coefficients A of the homogeneous system is given by 



A = 





1 


2 


4 


1 


-1 


3 


-1 


1 


7 


-11 


13 


.11 


1 


12 


10 



The first thing we must do is determine r(A). Since the proof of Theorem 3.13 
dealt with columns, we choose to construct a new matrix B by applying ele- 
mentary column operations to A. Thus we define 



B = 



( 1 








0^ 


-1 


4 


5 


3 


7 


-20 


-25 


-15 


. 1 


8 


10 


6. 



where the columns of B are given in terms of those of A by B^ = A'^, B^ = 
A^ - 3A^, B^ = A^ - 2A^ and B"^ = A"^ - 4A^ . It is obvious that B^ and B^ are 
independent, and we also note that B^ = (5/4)B^ and B"^ = (3/4)B^. Then 
r(A) = r(B) = 2, and hence we have dim S = 4 - 2 = 2. 

(An alternative method of finding r(A) is as follows. If we interchange the 
first two rows of A and then add a suitable multiple the new first row to elimi- 
nate the first entry in each of the remaining three rows, we obtain 

'I -1 3 -V 

4-77 

8 -14 14 

,0 12 -21 21, 
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It is now clear that the first two rows of this matrix are independent, and that 
the third and fourth rows are each multiples of the second. Therefore r(A) = 2 
as above.) 

We now follow the first part of the proof of Theorem 3.13. Observe that 
since r(A) = 2 and the first two columns of A are independent, we may write 

A^ = (5/4)A1 - (7/4)A2 

and 

A'^ = (3/4)A1 + (7/4)A2 . 
We therefore define the vectors 

b(3) = (5/4,-7/4,-1,0) 

and 

b^'^^ = (3/4,7/4,0,-1) 

which are independent solutions of the homogeneous system and span the 
solution space S. Therefore the general solution set to the nonhomogeneous 
system is given by 

M + S = {M + aM3)+)3&(4)} 

= {(2, 5, 1, -3) + a(5 1 A, -II 4, -1, 0) + /3(3 / 4, 7 / 4, 0, 1)} 
where a, (3 £ IR are arbitrary. / 



Exercises 

1. Find the dimension and a basis for the solution space of each of the fol- 
lowing systems of linear equations over IR: 

(a) x + 4>' + 2z = {b) x + 3y+2z = 

lx+ y + 5z = x + 5y+z = 

3x + 5j + 8z = 



(c) x + 2y + 2z-w + 3t = Q 
x + ly + 'iz + w+ t = Q 
3x + 6y + ^z + w+ t = Q 



(d) x + 2y-2z-2w- t = 
x + 2y- z+3w-2t = 
2x + Ay-lz+ w+ t = Q 
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2. Consider the subspaces U and V of IR given by 

U = {(a, b,c,d)^R'^:b + c + d = 0} 

V = {(a,b,c,d)^R'^:a + b = 0mdc = 2d} . 

(a) Find the dimension and a basis for U. 

(b) Find the dimension and a basis for V. 

(c) Find the dimension and a basis for U fl V. 

3. Find the complete solution set of each of the following systems of linear 
equations over IR: 

(a) 3x - y = 7 (b) 2x - y + 3z = 5 

2x + y = I 3x + 2y-2z= I 

lx+ 4z = ll 

(c) 5x + 2y- z = (d) x- y+2z+ w = 3 

3jc + 53;+3z = 2x+ y- z- w = l 

jc + 83; + 7z = 3x+ y+ z-3w = 2 

3x-2y + 6z =7 



3.6 MATRIX ALGEBRA 

We now introduce the elementary algebraic operations on matrices. These 
operations will be of the utmost importance throughout the remainder of this 
text. In Chapter 5 we will see how these definitions arise in a natural way 
from the algebra of linear transformations. 

Given two m x n matrices A = (aij) and B = (bij), we define their sum A + 
B to be the matrix with entries 

(A + B)ij = aij + bij 

obtained by adding the corresponding entries of each matrix. Note that both A 
and B must be of the same size. We also say that A equals B if a^ = by for all 
i and j. It is obvious that 

A + B = B + A 

and that 

A + (B + C) = (A + B) + C 
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for any other m x n matrix C. We also define the zero matrix as that matrix 
for which A + = A. In other words, (0)ij = for every i and j. Given a matrix 
A = (aij), we define its negative (or additive inverse) 

-A = (-ay) 

such that A + (-A) = 0. Finally, for any scalar c we define the product of c and 
A to be the matrix 

cA = (caij) . 

Since in general the entries ajj in a matrix A = (a^) are independent of each 
other, it should now be clear that the set of all m x n matrices forms a vector 
space of dimension mn over a field ^ of scalars. In other words, any m x n 
matrix A with entries a^ can be written in the form 

m n 
i=l J=l 

where the m x n matrix Ejj is defined as having a 1 in the (i, i)th position and 
O's elsewhere, and there are clearly mn such matrices. We denote the space of 
all m x n matrices over the field by Minxn(^- The particular case of m = n 
defines the space M^(Jf) of all square matrices of size n. We will often refer 
to a matrix in MnClF) as an n-square matrix. 

Now let A e MmxnClD be an m X n matrix, B e MrxmCiO be an r x m 
matrix, and consider the two systems of linear equations 

n 

^^ij^j=yi' i = l,...,m 

7=1 

and 

m 

7=1 

where X = (x„ . . . , Xn) G J", Y = (y„ . . . , y^) G J"^ and Z = (z„ . . . , z^) G 
Substituting the first of these equations into the second yields 

Zi = 2jbijyj = 2jbij2k3]kXk = 2kCikXk 

where we defined the product of the r x m matrix B and the m x n matrix A to 
be the r x n matrix C = B A whose entries are given by 

m 

7 = 1 

Thus the (i, k)th entry of C = BA is given by the standard scalar product 
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(BA)i, = B-A'^ 

of the ith row of B with the kth column of A (where both are considered as 
vectors in ^™). Note that matrix multiplication is generally not commutative, 
i.e., AB ^ BA. Indeed, the product AB may not even be defined. 

Example 3.10 Let A and B be given by 





(1 


6 


-2^ 




(2 -9\ 


A = 


3 


4 


5 


B = 


6 1 




J 





8. 




.1 -3. 



Then the product of A and B is given by 



(I 



C = AB = 



3 4 
7 



6 -2^ 
5 
8 



/2 -9^ 
6 1 

Vl -3 



/1-2 + 6-6-2-1 -l-9 + 6-l + 2-3^ 
3-2 + 4-6 + 5-1 -3-9 + 4-1-5-3 
7-2 + 0-6 + 8-1 -7-9 + 0-1-8-3 



^36 3^ 
35 -38 
^22 -87 



Note that it makes no sense to evaluate the product B A. 
It is also easy to see that if we have the matrices 



A = 




and B = 




then 



AB = 



(I 2\ 










.3 4, 


(; 


0. 


■c 





while 



BA = 







(1 




'3 4\ 


.1 


:) 


.3 




.1 V 



Example 3.11 Two other special cases of matrix multiplication are worth ex- 
plicitly mentioning. Let X £ be the column vector 
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If A is an m X n matrix, we may consider X to be an n x 1 matrix and form the 
product AX: 



/ajj ••• a 



AX = 



In 



( X, 



As expected, the product AX is an m x 1 matrix with entries given by the stan- 
dard scalar product Ai»X in of the ith row of A with the vector X. Note 
that this may also be written in the form 



AK = 



hi 



\^ml/ 



la \ 



In 



which clearly shows that AX is just a linear combination of the columns of A. 

Now let Y G ^™ be the row vector Y = (y,, . . . , ym). If we view this as a 
1 X m matrix, then we may form the 1 x n matrix product YA given by 



YA = {y^, ... , yj 



Hi 



\^ml 



a 



In 



= (yi«ii + ••• + yAv 

= (F»Ai, ... , F»A«) 



This again yields the expected form of the product with entries Y» A\ / 

This example suggests the following commonly used notation for systems 
of linear equations. Consider the system 

n 

where A = (ay) is an m x n matrix. Suppose that we define the column vectors 
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J° and Y = 



Vm) 



If we consider X to be an n x 1 matrix and Y to be an m x 1 matrix, then we 
may write this system in matrix notation as 

AX = Y . 

Note that the ith row vector of A is Aj = (aii, . . . , ain) so that the expression 
2jaijXj = yi may be written as the standard scalar product 

Ai . X = y; . 

We leave it to the reader to show that if A is an n x n matrix, then 

AIn = InA = A . 

Even if A and B are both square matrices (i.e., matrices of the form m x m), 
the product AB will not generally be the same as BA unless A and B are 
diagonal matrices (see Exercise 3.6.4). However, we do have the following. 

Theorem 3.17 For matrices of proper size (so that these operations are 
defined), we have: 

(associative law), 
(left distributive law), 
(right distributive law). 



(a) (AB)C = A(BC) 

(b) A(B + C) = AB + AC 

(c) (B + C)A = BA + CA 



(d) k(AB) = (kA)B = A(kB) for any scalar k. 



Proof (a) [(AB)C\j = 2,(A5),,q^. = 2,^ MM^kj = ^r, k^iriKkC,,) 
= 2,a,,(5C),^.=[A(5C)],^ . 



(b) VA{B + C)\ = {B + C\ = {b,^ + c,^ ) 
= [(AB) + (AC)]y . 



(c) Left to the reader (Exercise 3.6. 1). 

(d) Left to the reader (Exercise 3.6. 1). I 
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Given a matrix A = (aij), we define the transpose of A, denoted by A^ = 
(a^ij) to be the matrix with entries given by a\ = aji. In other words, if A is an 

m X n matrix, then A^ is an n x m matrix whose columns are just the rows of 
A. Note in particular that a column vector is just the transpose of a row vector. 



Example 3.12 If A is given by 




then A^ is given by 



(1 4\ 

2 5 

3 6 



/ 



Theorem 3.18 The transpose has the following properties: 

(a) (A + B)'^ = A^ + B'r. 

(b) (AT)T = a. 

(c) (cA)^ = cA^ for any scalar c. 

(d) (AB)T = B'rA'r. 

Proof (a) [(A + B)^];^ = [(A + B)l, = a,, + bj; = + b\. = (A^ + b\. 

(b) (AT)T.. = (AT)ji = aij = (A)ij. 

(c) (cA)\. = (cA)ji = caji = c(AT),j. 

(d) (AB)T. = (AB)j, = 2k^kKi = 2kb\a\j = (BTAT)... I 

We now wish to relate this algebra to our previous results dealing with the 
rank of a matrix. Before doing so, let us first make some elementary observa- 
tions dealing with the rows and columns of a matrix product. Assume that 
A e MmxnCD and B e MnxrCD so that the product AB is defined. Since the 
(i, i)th entry of AB is given by (AB)ij = Skaikbkj, we see that the ith row of AB 
is given by a linear combination of the rows of B: 

(AB)i = (2kaikbu, . . . , 2kaikbkr) = 2kaik(bki, . . . , bkr) = 2kaikBk . 



Another way to write this is to observe that 
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(bn - h\ 

= (an,...,ai„) 



nr I 



Similarly, for the columns of a product we find that the '^th column of AB is a 
linear combination of the columns of A: 



and 



'^k^ikhki 


n 


la \ 
"lit 






-1 




b, =\^'b,, 


^k'^mk^kj ^ 


k=l 


\^mki 


k=\ 



{AS)} = 



'^k(^\kbki^ 



\^k^mkbkj I 



a 



11 



a 



mn I 



b 



These formulas will be quite useful to us in several of the following theorems. 
Theorem 3.19 For any matrix A we have r(A^) = r(A). 
Proof This is Exercise 3.6.2. I 

Theorem 3.20 If A and B are any matrices for which the product AB is 
defined, then the row space of AB is a subspace of the row space of B, and the 
column space of AB is a subspace of the column space of A. 

Proof Using (AB)i = SkaikBk, we see that the ith row of AB is in the space 
spanned by the rows of B, and hence the row space of AB is a subspace of the 
row space of B. 

Now note that the column space of AB is just the row space of (AB)^ = 
B^A^, which is a subspace of the row space of A^ by the first part of the 
theorem. But the row space of A^ is just the column space of A. I 

Corollary r(AB) < min{r(A), r(B)}. 

Proof Let Va be the row space of A, and let Wa be the column space of A. 
Then 

r(AB) = dimVAB ^ dimVe = r(B) 
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while 

r(AB) = dimWAB ^ dimWA = r(A) . I 

Exercises 

1 . Complete the proof of Theorem 3.17. 

2. Prove Theorem 3.19. 

3. Let A be any m x n matrix and let X be any n x 1 matrix, both with 
entries in 5. Define the mapping f : J" by f(X) = AX. 

(a) Show that f is a linear transformation (i.e., a vector space homomor- 
phism). 

(b) Define Im f = {AX: X e 5°}. Show that Im f is a subspace of J"'. 

(c) Let U be the column space of A. Show that Im f = U. [Hint: Use 

Example 3.11 to show that Im f C U. Next use the equation (AI)J = AI^ to 
show that U C Im f.] 

(d) Let N denote the solution space to the system AX = 0. In other 
words, N = {X e iF": AX = 0}. (N is usually called the null space of A.) 
Show that 

dim N + dim U = n . 

[Hint: Suppose dim N = r, and extend a basis {xi, . . . , Xr} for N to a 
basis {Xi} for Show that U is spanned by the vectors Ax^+i , . . . , 
Axn , and then that these vectors are linearly independent. Note that this 
exercise is really just another proof of Theorem 3.13.] 

4. A matrix of the form 

'ail ••• ' 
••• 

.0 - a„„^ 

is called a diagonal matrix. In other words, A = (ajj) is diagonal if ajj = 
for i j. If A and B are both square matrices, we may define the commu- 
tator [A, B] of A and B to be the matrix [A, B] = AB - BA. If [A, B] = 0, 
we say that A and B commute. 

(a) Show that any diagonal matrices A and B commute. 
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(b) Prove that the only n x n matrices which commute with every n x n 
diagonal matrix are diagonal matrices. 



5. 
6. 



Given the matrices 



( 2 



A = 



1 

-3 



-W 


4/ 



B = 



(I -2 
.3 4 



compute the following: 

(a) AB. 

(b) BA. 

(c) AA^. 

(d) A^A. 

(e) Verify that (AB)'^ = bV. 



6. Consider the matrix A £ MnCF) given by 



A = 



(0 I Q Q 
10 
1 








Q\ 





1 




Thus A has zero entries everywhere except on the superdiagonal where 
the entries are 1 's. Let A^ = AA, A^ = AAA, and so on. Show that A° = 
but A°-i9£0. 

7. Given a matrix A = (ajj) G Mn(^, the sum of the diagonal elements of A 
is called the trace of A, and is denoted by Tr A. Thus 

1=1 

(a) Prove that Tr(A + B) = Tr A + Tr B and that Tr(aA) = a(Tr A) for 
any scalar a. 

(b) Prove that Tr(AB) = Tr(BA). 

8. (a) Prove that it is impossible to find matrices A, B E Mn(IR) such that 
their commutator [A, B] = AB - BA is equal to 1. 
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(b) Let ^ be a field of characteristic 2 (i.e., a field in which 1 + 1 = 0; see 
Exercise 1.5.17). Prove that it is possible to find matrices A, B £ y^ii^F) 
such that [A, B] = 1. 

9. A matrix A = (ay) is said to be upper-triangular if ajj = for i > j. In 
other words, every entry of A below the main diagonal is zero. Similarly, 
A is said to be lower- triangular if aij = for i < j. Prove that the product 
of upper (lower) triangular matrices is an upper (lower) triangular matrix. 



10. Consider the so-called Pauli spin matrices 

/'O -A 



o, = 



(0 l\ 
.1 



/ 




and define the permutation symbol eyk by 



1 if (j, j, k) is an even permutation of (1, 2, 3) 
-1 if (i, j, k) is an odd permutation of (1, 2, 3) 
if any two indices are the same 



The commutator of two matrices A, B G M^i^f) is defined by [A, B] = 
AB - B A, and the anticommutator is given by [A, B] + = AB + B A. 

(a) Show that [Oi, oj] = 2i 2k£ijk Ok- In other words, show that OiOj = iOk 
where (i, j, k) is an even permutation of (1, 2, 3). 

(b) Show that [O;, 0,]+ = 2I8ij . 

(c) Using part (a), show that Tr Oj = 0. 

(d) For notational simplicity, define Oq = 1. Show that {oq, 0\, 02, 03} 
forms a basis for M2(C). [Hint: Show that Tr(aa o p) = 26a|5 where < a, 
|3 < 3. Use this to show that {Oa} is linearly independent.] 

(e) According to part (d), any X G M2(C) may be written in the form X = 
2aXaOa- How would you find the coefficients Xa? 

(f) Show that (Oa, crp) = (l/2)Tr(aaCrp) defines an inner product on 
M2(C). 

(g) Show that any matrix X G M2(C) that commutes with all of the o, 
(i.e., [X, Oi] = for each i = 1, 2, 3) must be a multiple of the identity 
matrix. 
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11. 



A square matrix S is said to be symmetric if = S, and a square matrix 
A is said to be skewsymmetric (or antisymmetric) if A^ = -A. (We 

continue to assume as usual that ^ is not of characteristic 2.) 

(a) Show that S ;^ and A are linearly independent in M^(Jf). 

(b) What is the dimension of the space of all n x n symmetric matrices? 

(c) What is the dimension of the space of all n x n antisymmetric 
matrices? 



12. Find a basis {Aj} for the space MnClT) that consists only of matrices with 

the property that Aj^ = Aj (such matrices are called idempotent or 
projections). [Hint: The matrices 







will work in the particular case of M2(^.] 

13. Show that it is impossible to find a basis for MnCiF) such that every pair 
of matrices in the basis commute with each other. 

14. (a) Show that the set of all nonsingular n x n matrices forms a spanning 
set for Mn(^. Exhibit a basis of such matrices. 

(b) Repeat part (a) with the set of all singular matrices. 

15. Show that the set of all matrices of the form AB - BA do not span 
MnClF). [Hint. Use the trace.] 



16. Is it possible to span MnCJTO using powers of a single matrix A? In other 

words, can {In , A, A^, . . . , A", . . .} span MnClTO? [Hint: Consider 
Exercise 4 above.] 



3.7 INVERTIBLE MATRICES 

We say that a matrix A G Mn(^ is nonsingular if r(A) = n, and singular if 
r(A) < n. Given a matrix A G Mn(^, if there exists a matrix B G Mn(!f) such 
that AB = BA = In, then B is called an inverse of A, and A is said to be 
invertible. 

Technically, a matrix B is called a left inverse of A if BA = I, and a 
matrix B' is a right inverse of A if AB' = 1. Then, if AB = BA = I, we say that 
B is a two-sided inverse of A, and A is then said to be invertible. 
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Furthermore, if A has a left inverse B and a right inverse B', then it is easy to 
see that B = B' since B = Bl = B(AB') = (BA)B' = IB' = B'. We shall now 
show that if B is either a left or a right inverse of A, then A is invertible. 

Theorem 3.21 A matrix A £ MnCF) has a right (left) inverse if and only if A 
is nonsingular. This right (left) inverse is also a left (right) inverse, and hence 
is an inverse of A. 

Proof Suppose A has a right inverse B. Then AB = In so that r(AB) = r(ln). 
Since r(ln) is clearly equal to n (Theorem 3.9), we see that r(AB) = n. But then 
from the corollary to Theorem 3.20 and the fact that both A and B are n x n 
matrices (so that r(A) < n and r(B) < n), it follows that r(A) = r(B) = n, and 
hence A is nonsingular. 

Now suppose that A is nonsingular so that r(A) = n. If we let EJ be the ith 
column of the identity matrix In, then for each j = 1, . . . , n the system of 
equations 



has a unique solution which we denote by X = B^ (Theorem 3.16). Now let B 
be the matrix with columns B^. Then the ith column of AB is given by 



and hence AB = In. It remains to be shown that BA = In. To see this, note that 
r(A^) = r(A) = n (Theorem 3.19) so that A^ is nonsingular also. Hence apply- 
ing the same argument shows there exists a unique n x n matrix such that 

A'^C'^ = In. Since (CA)"^ = A'^C'^ and In^ = In, this is the same as CA = In. 
We now recall that it was shown prior to the theorem that if A has both a left 
and a right inverse, then they are the same. Therefore B = C so that BA = AB 
= In, and hence B is an inverse of A. Clearly, the proof remains valid if "right" 
is replaced by "left" throughout. I 

Corollary 1 A matrix A El MnCiF) is nonsingular if and only if it has an 
inverse. Furthermore, this inverse is unique. 

Proof As we saw above, if B and C are both inverses of A, then B = Bl = 
B(AC) = (BA)C = IC = C. I 

In view of this corollary, the unique inverse to a matrix A will be denoted 
by A"' from now on. 



n 




(AB)J = ABJ = EJ 
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Corollary 2 If A is an n x n nonsingular matrix, then A ^ is nonsingular and 
(A-')-i = A. 

Proof If A is nonsingular, then (by Theorem 3.21) A"' exists so that A"'A = 
AA"^ = I. But this means that (A"')"^ exists and is equal to A, and hence A"' is 
also nonsingular. I 

Corollary 3 If A and B are nonsingular then so is AB, and (AB)"' = B"' A"' . 

Proof The fact that A and B are nonsingular means that A"' and B"' exist. 
We therefore see that 

(B-'A-')(AB) = B-'IB = B-'B = I 

and similarly (AB)(B-'A-') = 1. It then follows that B"' A"' = (AB)"'. Since we 
have now shown that AB has an inverse. Theorem 3.21 tells us that AB must 
be nonsingular. I 

Corollary 4 If A is nonsingular then so is PJ, and {PJy = (A-i)^. 

Proof That A^ is nonsingular is a direct consequence of Theorem 3.19. Next 
we observe that 

(A-')TaT = (AA-')^ = 1^ = I 

so that the uniqueness of the inverse tells us that (A^)"' = (A"')^. Note this 
also shows that A^ is nonsingular. I 

Corollary 5 A system of n linear equations in n unknowns has a unique 
solution if and only if its matrix of coefficients is nonsingular. 

Proof Consider the system AX = Y. If A is nonsingular, then a unique A"' 
exists, and therefore we have X = A"'Y as the unique solution. (Note that this 
is essentially the content of Theorem 3.16.) 

Conversely, if this system has a unique solution, then the solution space of 
the associated homogeneous system must have dimension (Theorem 3.15). 
Then Theorem 3.13 shows that we must have r(A) = n, and hence A is non- 
singular. I 

A major problem that we have not yet discussed is how to actually find the 
inverse of a matrix. One method involves the use of determinants as we will 
see in the next chapter. However, let us show another approach based on the 
fact that a nonsingular matrix is row-equivalent to the identity matrix 
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(Theorem 3.10). This method has the advantage that it is algorithmic, and 
hence is easily implemented on a computer. 

Since the jth column of a product AB is AB-*, we see that considering the 
particular case of AA"' = I leads to 

(AA-')j = A(A-')j = EJ 

where EJ is the jth column of I. What we now have is the nonhomogeneous 
system 

AX = Y 



(or SjaijXj = yi) where X = (A"')-' and Y = EJ. As we saw in Section 3.2, we 
may solve for the vector X by reducing the augmented matrix to reduced row- 
echelon form. For the particular case of j = 1 we have 



aug A = 



a 



21 



«2„ 



and hence the reduced form will be 



'10 ••• Cii' 
1 ••• 

.0 - 1 c„i^ 



for some set of scalars Cij. This means that the solution to the system is Xi = 

Cu, X2 = C21, . . . , Xn = Cm. But X = (A"')^ = the first column of A"', and 
therefore this last matrix may be written as 



a 







-1 \ 
11 



1 a- 



nlj 



Now, for each j = 1, . . . , n the system AX = A(A"')j = E^ always has the 

same matrix of coefficients, and only the last column of the augmented matrix 
depends on j. Since finding the reduced row-echelon form of the matrix of 



3.7 INVERTIBLE MATRICES 



161 



coefficients is independent of this last column, it follows that we may solve all 
n systems simultaneously by reducing the single matrix 



Hi 



a 



In 



a„ 



1 - 0^ 
••• 1 



In other words, the reduced form will be 
'\ ■■■ I a- 







1 



11 



nl 



-1 \ 

In 



a' 



where the matrix A ' = (a Sj) satisfies AA ' = I since (AA = A(A ')j = is 
satisfied for each j = 1, . . . , n. 

Example 3.13 Let us find the inverse of the matrbc A given by 



/-I 2 n 

3-2 
^2-1 



We leave it as an exercise for the reader to show that the reduced row-echelon 
form of 



-1 


2 


1 


1 











3 


-2 





1 





2 


-1 











1 



IS 











1/6 


1/12 


7/12 





1 





1/3 


1/6 


1/6 








1 


1/2 


-1/4 


1/4 



and hence A is given by 



/1/6 
1/3 
^1/2 -1/4 



1/12 7/12^ 
1/6 1/6 



1/4 



/ 
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Exercises 

1. Verify the reduced row-echelon form of the matrix given in Example 
3.13. 

2. Find the inverse of a general 2x2 matrix. What constraints are there on 
the entries of the matrix? 

3. Show that a matrix is not invertible if it has any zero row or column. 

4. Find the inverse of each of the following matrices: 





(1 





2^ 




(1 3 


4^ 




^1 


2 1\ 


(a) 


2 


-1 


3 


(b) 


3 -1 


6 


(c) 


2 


5 2 




.4 


1 


8. 




ri 5 


1. 




.1 


3 3^ 



5. Use the inverse of the matrix in Exercise 4(c) above to find the solutions 
of each of the following systems: 

(a) x + 2y+ z = lO (b) x + 2y+ z= 2 
2x + 5y + 2z = U 2x + 5y + 2z = -l 

x + 3y+3z = 30 x + 3y+3z= 6 

6. What is the inverse of a diagonal matrix? 

7. (a) Prove that an upper-triangular matrix is invertible if and only if every 
entry on the main diagonal is nonzero (see Exercise 3.6.9 for the defini- 
tion of an upper- triangular matrix). 

(b) Prove that the inverse of a lower (upper) triangular matrix is lower 
(upper) triangular. 

8. Find the inverse of the following matrix: 

'1 2 3 4' 

2 3 4 

3 4' 

^0 4^ 



9. (a) Let A be any 2x1 matrix, and let B be any 1x2 matrix. Prove that 
AB is not invertible. 
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(b) Repeat part (a) where A is any m x n matrix and B is any n x m 
matrix with n < m. 

10. Summarize several of our results by proving the equivalence of the fol- 
lowing statements for any n x n matrix A: 

(a) A is invertible. 

(b) The homogeneous system AX = has only the zero solution. 

(c) The system AX = Y has a solution X for every n x 1 matrix Y. 

11. Let A and B be square matrices of size n, and assume that A is 
nonsingular. Prove that r(AB) = r(B) = r(BA). 

12. A matrix A is called a left zero divisor if there exists a nonzero matrix B 
such that AB = 0, and A is called a right zero divisor if there exists a 
nonzero matrix C such that CA = 0. If A is an m x n matrix, prove that: 

(a) If m < n, then A is a left zero divisor. 

(b) If m > n, then A is a right zero divisor. 

(c) If m = n, then A is both a left and a right zero divisor if and only if A 
is singular. 

13. Let A and B be nonsingular symmetric matrices for which AB - BA = 0. 
Show that AB, A"'B, AB"' and A 'B"' are all symmetric. 

3.8 ELEMENTARY MATRICES 

Recall the elementary row operations a, (3, y described in Section 3.2. We now 
let e denote any one of these three operations, and for any matrix A we define 
e(A) to be the result of applying the operation e to the matrix A. In particular, 
we define an elementary matrix to be any matrix of the form e(I). The great 
utility of elementary matrices arises from the following theorem. 

Theorem 3.22 If A is any m x n matrix and e is any elementary row opera- 
tion, then 

e(A) = e(U)A . 

Proof We must verify this equation for each of the three types of elementary 
row operations. First consider an operation of type a. In particular, let a be 
the interchange of rows i and j. Then 



[e(A)]k = Ak fork^ii, j 
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while 

[e(A)]i = Aj and [e(A)]j = A; . 
On the other hand, using (AB)k = Aj, B we also have 

[e(I)A]k = [e(I)]kA . 
If k i, j then [e(I)]k = Ik so that 

[e(I)]kA = IkA = Ak . 

Ifk = i,then[e(I)]i = Ij and 

[e(I)]iA = IjA = Aj . 

Similarly, we see that 

[e(I)]jA = liA = Ai . 

This verifies the theorem for transformations of type a. (It may be helpful for 
the reader to write out e(I) and e(I)A to see exactly what is going on.) 

There is essentially nothing to prove for type p transformations, so we go 
on to those of type y. Hence, let e be the addition of c times row j to row i. 
Then 

[e(I)]k= Ik fork^ti 

and 

[e(I)]i = li + clj . 

Therefore 

[e(I)]iA = (Ii + cIj)A = Ai + cAj = [e(A)]i 
and for k 9i i we have 

[e(I)]kA = IkA = Ak = [e(A)]k . I 

If e is of type a, then rows i and j are interchanged. But this is readily 
undone by interchanging the same rows again, and hence e"' is defined and is 
another elementary row operation. For type (3 operations, some row is multi- 
plied by a scalar c, so in this case e"' is simply multiplication by 1/c. Finally, a 
type Y operation adds c times row j to row i, and hence e"^ adds -c times row j 
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to row i. Thus all three types of elementary row operations have inverses 
which are also elementary row operations. 

By way of nomenclature, a square matrix A = (ajj) is said to be diagonal if 
aij = for i j. The most common example of a diagonal matrix is the identity 
matrix. 

Theorem 3.23 Every elementary matrix is nonsingular, and 

[e(l)]-' = e-^I) . 

Furthermore, the transpose of an elementary matriK is an elementary matrix. 

Proof By definition, e(I) is row equivalent to I and hence has the same rank 
as 1 (Theorem 3.4). Thus e(l) is nonsingular since r(ln) = n, and hence e(l)"' 
exists. Since it was shown above that e"' is an elementary row operation, we 
apply Theorem 3.22 to the matrix e(l) to obtain 

e-'(I)e(I) = e-'(e(I)) = I . 

Similarly, applying Theorem 3.22 to e"'(l) yields 

e(l)e-'(l) = e(e-'(l)) = 1 . 

This shows that e-'(I) = [e(I)]-'. 

Now let e be a type a transformation that interchanges rows i and j (with 
i < j). Then the ith row of e(l) has a 1 in the ]th column, and the ]th row has a 
1 in the ith column. In other words, 

[e(l)]ij = 1 = [e(l)]ji 

while for r, s 9^ i, j we have 

[e(l)]rs=0 ifr^s 

and 

[e(I)]rr= 1 . 

Taking the transpose shows that 

[e(l)]T,j= [e(l)V= 1 = [e(l)]ij 

and 

[e(l)]\s= [e(l)]sr= = [e(l)]rs . 
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Thus [e(I)] = e(I) for type a operations. 

Since I is a diagonal matrix, it is clear that for a type |3 operation which 

simply multiplies one row by a nonzero scalar, we have [e(l)]T = e(I). 

Finally, let e be a type y operation that adds c times row j to row i. Then 
e(I) is just I with the additional entry [e(I)]ij = c, and hence [e(I)]^ is just I 
with the additional entry [e(I)]ji = c. But this is the same as c times row i 

added to row j in the matrix I. In other words, [e(I)] ^ is just another elemen- 
tary matrix. I 

We now come to the main result dealing with elementary matrices. For 
ease of notation, we denote an elementary matrix by E rather than by e(I). In 
other words, the result of applying the elementary row operation ei to I will be 
denoted by the matrix Ej = ei(I). 

Theorem 3.24 Every nonsingular n x n matrix may be written as a product 
of elementary n x n matrices. 

Proof It follows from Theorem 3.10 that any nonsingular n x n matrix A is 
row equivalent to In. This means that In may be obtained by applying r suc- 
cessive elementary row operations to A. Hence applying Theorem 3.22 r times 
yields 

Er • • • EiA = In 

SO that 

.A. — E]^ • • • Ej- Ij^ — E]^ • • • Ej* . 

The theorem now follows if we note that each Ej"' is an elementary matrix ac- 
cording to Theorem 3.23 (since Ej"' = [e(I)]"' = e"^(I) and e"' is an elementary 
row operation). I 

Corollary If A is an invertible n x n matrix, and if some sequence of ele- 
mentary row operations reduces A to the identity matrix, then the same 
sequence of row operations reduces the identity matrix to A"' . 

Proof By hypothesis we may write Er • • • E,A = I. But then multiplying from 
the right by A"' shows that A"' = Er • • • EJ. I 

Note this corollary provides another proof that the method given in the 
previous section for finding A"' is valid. 

There is one final important property of elementary matrices that we will 
need in a later chapter. Let E be an n x n elementary matrix representing any 
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of the three types of elementary row operations, and let A be an n x n matrix. 
As we have seen, multiplying A from the left by E results in a new matrix 
with the same rows that would result from applying the elementary row 
operation to A directly. We claim that multiplying A from the right by 
results in a new matrix whose columns have the same relationship as the rows 
of EA. We will prove this for a type y operation, leaving the easier type a and 
|3 operations to the reader (see Exercise 3.8.1). 

Let Y be the addition of c times row j to row i. Then the rows of E are 
given by Ek = Ik for k^i, and Ei = Ij + clj. Therefore the columns of E^ are 
given by 

(eY = 1*' fork^ti 

and 

(ET)i = r + ciJ . 

Now recall that the kth column of AB is given by (AB)*^ = AB'^. We then have 
(AE^)'' = A(E'^)'^ = AI'^ = a'' for k ?£ i 

and 

(AE'^y = A(ET)i = A(r + cJj) = AI^ + cAjj = A^ + cA^ . 

This is the same relationship as that found between the rows of EA where 
(EA)k = Ak and (EA)i = Aj + cAj (see the proof of Theorem 3.22). 

Exercises 

1. Let A be an n X n matrix, and let E be an n x n elementary matrix repre- 
senting a type a or p operation. Show that the columns of AE^ have the 
same relationship as the rows of EA. 

2. Write down 4x4 elementary matrices that will induce the following ele- 
mentary operations in a 4 x 4 matrix when used as left multipliers. Verify 
that your answers are correct. 

(a) Interchange the 2nd and 4th rows of A. 

(b) Interchange the 2nd and 3rd rows of A. 

(c) Multiply the 4th row of A by 5. 

(d) Add k times the 4th row of A to the 1st row of A. 

(e) Add k times the 1st row of A to the 4th row of A. 

3. Show that any ea(A) may be written as a product of e|3(A)'s and eY(A)'s. 
(The notation should be obvious.) 
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Pick any 4x4 matrix A and multiply it from the right by each of the ele- 
mentary matrices found in the previous problem. What is the effect on A? 

Prove that a matrix A is row equivalent to a matrix B if and only if there 
exists a nonsingular matrix P such that B = PA. 



Reduce the matrix 



A = 



/I 

2 



2^ 
3 -1 
3 3j 



to the reduced row-echelon form R, and write the elementary matrix cor- 
responding to each of the elementary row operations required. Find a 
nonsingular matrix P such that PA = R by taking the product of these ele- 
mentary matrices. 

7. Let A be an n X n matrix. Summarize several of our results by proving 
that the following are equivalent: 

(a) A is invertible. 

(b) A is row equivalent to In . 

(c) A is a product of elementary matrices. 

8. Using the results of the previous problem, prove that if A = A, A2 • • • At 
where each Aj is a square matrix, then A is invertible if and only if each 
of the Ai is invertible. 

The remaining problems are all connected, and should be worked in the given 
order. 



9. Suppose that we define elementary column operations exactly as we did 
for rows. Prove that every elementary column operation on A can be 
achieved by multiplying A on the right by an elementary matrix. [Hint: 
You can either do this directly as we did for rows, or by taking transposes 
and using Theorem 3.23.] 

10. Show that an m x n reduced row-echelon matrix R of rank k can be 
reduced by elementary column operations to an m x n matrix C of the 
form 
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C = 



(I 

1 












1 





0\ 







0; 



where the first k entries on the main diagonal are I's, and the rest are O's. 

11. From the previous problem and Theorem 3.3, show that every m x n 
matrix A of rank k can be reduced by elementary row and column opera- 
tions to the form C. We call the matrix C the canonical form of A. 

12. We say that a matrix A is row- column- equivalent (abbreviated by r.c.e.) 
to a matrix B if A can be transformed into B by a finite number of ele- 
mentary row and column operations. Prove: 

(a) If A is a matrix, e is an elementary row operation, and e' is an 
elementary column operation, then (eA)e' = e(Ae'). 

(b) r.c.e. is an equivalence relation. 

(c) Two m X n matrices A and B are r.c.e. if and only if they have the 
same canonical form, and hence if and only if they have the same rank. 

13. If A is any m x n matrix of rank k, prove that there exists a nonsingular 
m X m matrix P and a nonsingular n x n matrix Q such that PAQ = C (the 
canonical form of A). 

14. Prove that two m x n matrices A and B are r.c.e. if and only if there exists 
a nonsingular m x m matrix P and a nonsingular n x n matrix Q such that 
PAQ = B. 



CHAPTER 4 



Determinants 



Suppose we want to solve the system of equations 

ax + by = f 
cx + dy = g 

where a, b, c, d, f, g G ^. It is easily verified that if we reduce the augmented 
matrix to reduced row-echelon form we obtain 

1 (fd-gb)/A\ 
1 (ag-cf)/A) 

where A = ad - cb. We must therefore have A ;t if a solution is to exist for 
every choice of f and g. If A E M2CD is the matrix of coefficients of our sys- 
tem, we call the number A the determinant of A, and write this as det A. 
While it is possible to proceed from this point and define the determinant of 
larger matrices by induction, we prefer to take another more useful approach 
in developing the general theory. We will find that determinants arise in many 
different and important applications. Recall that unless otherwise noted, we 
always assume that ^ is not of characteristic 2 (see Exercise 1.5.15). 
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4.1 DEFINITIONS AND ELEMENTARY PROPERTIES 

Recalling our discussion of permutations in Section 1.2 we make the fol- 
lowing definition. If A = (aij) is an n x n matrix over a field J^, we define the 
determinant of A to be the scalar 

det A = 2^e5. (Sgn (T)aial«2a2 • • • «„an 



where ai is the image of i = 1, 
write the determinant as 



, n under the permutation a. We frequently 



detA = 



ni 



a 



nl 



in 



Note that our definition contains n! terms in the sum, where each term is a 
product of n factors ay , and where each of these terms consists of precisely 
one factor from each row and each column of A. The determinant of an n x n 
matrix A is said to be of order n. We will sometimes denote the determinant 
of A by |A|. Note that the determinant is only defined for a square matrix. 

Example 4.1 We leave it to the reader to show that in the case of a 2 x 2 
matrix, our definition agrees with the elementary formula 



a b 
c d 



= ad-cb 



In the case of a 3 x 3 matrix, we have 



det A = 2„(sgn o)a^^^a2^2^^^3 

^ ^12^23^31 ^ ^1 

~^13^22%l ~<^11^23%2 "^12^21^33 



- Cl^iCl22^33 + ^12*^23^1 ^13'^2l'^32 



The reader may recognize this from a more elementary course when written in 
the mnemonic form 
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Here, we are to add together all products of terms connected by a (+) line, and 
subtract all of the products connected by a (-) line. We will see in a later sec- 
tion that this 3x3 determinant may be expanded as a sum of three 2x2 
determinants. / 

Recall that a diagonal matrix A = (ajj) is defined by the property that a^ = 
for i^]. We therefore see that if A is any diagonal matrix, then 

n 

detA = aii ••• a„„=Y[au 

;=1 

since only the identity permutation results in solely nonzero factors (see also 
Theorem 4.5 below). In particular, we have the simple but important result 

detl = 1 . 

We now proceed to prove several useful properties of determinants. 
Theorem 4.1 For any A G Mn(f) we have det A^ = det A. 

Proof Consider one of the terms (sgn a)aioi • • • an on in det A. Then aioi is in 
the ith row and oith column of A, and as i varies from 1 to n, so does ai. If we 
write any factor a^ai as a^, then j = ai so that i = a"'j. By Theorem 1.5 we 
know that sgn o"' = sgn o, and hence we can write our term as 

(sgn a) aa 'i i • • • aa 'n n = (sgn 6) aei i • • • aen n 
where Q = o~K Therefore, since Sn = {6 = a"' : a G Sn}, we have 



det A = (sgn a)ai • • • a„ ^„ 
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But a\ = Oji so that 

det = (sgn 0)a^i • • • 
= 2ee5„(sgn0)fl0ii ••• 

and hence det A = det A^. I 

It will be very convenient for us to view the determinant of A G Mn(^ as 
a function of the row vectors Aj. When we wish to do this, we will write 

det A = det(Ai, . . . , An) . 

Now consider a matrix A G M^i^F) and assume that Ai = rBi + sCi where 
Bi = (bii, . . . , bin) and C, = (cu, . . . , cin) are any two arbitrary (row) vectors 
in and r, s G ^. We then have 

detA = det(Ai, ... , AJ 

= det(r5i+5Ci, A2, ... , AJ 

= 2aG5„ (sgn Cf){rb, „i + SC^ „i )a2 al'-'^nan 
= ^2aGS„ (sgn 0)bi „ia2 a2 • • a„ 

+^2aes„ (sgn cr)ci ^^a^ ■ ■ •«„ ^„ . 

If we now let B be the matrix such that Bj = Aj for i = 2, . . . , n and C be the 
matrix such that Ci = Aj for i = 2, . . . , n we see that 

det A = r det B + s det C . 

Generalizing slightly, we summarize this result as a theorem for easy refer- 
ence. 

Theorem 4.2 Let A G MnClF) have row vectors Ai, . . . , An and assume that 
for some i = 1, . . . , n we have 

Ai = rBi + sCi 

where Bj, Q G iF" and r, s G ^. Let B G MnCO have rows Ai, . . . , Ai-i , Bj, 
Ai+i , . . . , An and C G MnCO have rows Aj, . . . , Ai-i , Q, Ai+i , . . . , An. 
Then 

det A = r det B + s det C . 
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Corollary 1 Let A e MnClF) have rows Ai, . . . , An and suppose that for 
some i = 1, . . . , n we have 

7=1 

where Bj £ f or j = 1 , . . . , k and each rj £ J^. Then 

detA = det(Ai, ... , A^_„ ^^,rjBj, A,^„ ...,A„) 

k 

= 2)odet(Ai, ... , B-, A,.^i, ... , AJ . 
7=1 

Proof This follows at once by induction from the theorem. I 

Corollary 2 If any row of A G MnCT) is zero, then det A = 0. 

Proof If any row of A is zero, then clearly one factor in each term of the sum 
det A = 2aeSn(sgn a)aiai • • • anon will be zero. (This also follows from 
Theorem 4.2 by letting r = s = 0.) I 

Corollary 3 If A E MnCTO and r E J, then det(rA) = r" det A. 
Proof Since rA = (ray) we have 

det(rA) = 2^^^^ (sgn o){ra^ ^i) • • • (ra„ 

= r«detA . I 

For any A E MnClF) and a E Sn, we let oA denote the matrix with rows 
Acri , . . . , Aon- For example, if A and a are given by 



then 





(I 


2 


3\ 


A = 


4 


5 


6 




J 


8 


9. 



and 



o = 



'\ 2 3^ 
.3 1 2 





^7 


8 


9\ 


ctA = 


1 


2 


3 




4 


5 


6. 
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Theorem 4.3 For any A e MnCH and a £ Sn we have 

det(aA) = (sgno)detA . 

Proof First we note that by definition 

det(aA) = det(Aoi , . . . , Aon) = 2(i)es„ (sgn ^) aoi (j)i • • • aon (j)n • 

Now note that for each i = 1 , . . . , n there exists a j G { 1 , . . . , n} such that 
oj = i. Then j = a"'i and ^] = ^o'^i = 6i (where we have defined ^o'^ = 0) so 
that 

^oj (^j = ei • 

Since a is fixed we see that Sn = {6 = ^o'^: (j) G Sn}, and since by Theorem 
1.4 we have sgn (|) = sgn(6a) = (sgn 9)(sgn a), it follows that 

det(aA) = 2ees„ (sgn e)(sgn a) ai ei • • • an en = (sgn a)det A . I 

Note that all we really did in this last proof was rearrange the terms of 
det(aA) to where each product contained terms with the rows in natural (i.e., 
increasing) order. Since the sum is over all (|) G Sn, this is then the same as 
summing over all 9 G Sn where = ^o'\ 

Corollary 1 If B G MnC?0 is obtained from A G MnCiO by interchanging 
two rows of A, then det B = -det A. 

Proof If a is a transposition, then sgn a = -1 so that det B = det(aA) = 
-det A. I 

Corollary 2 If A G MnC?0 has two identical rows, then det A = 0. 

Proof If we let B be the matrix obtained by interchanging the two identical 
rows of A, then det A = det B = -det A implies that det A = 0. ■ 

Let us make two remarks. First, the reader should realize that because of 
Theorem 4.1, Theorems 4.2 and 4.3 along with their corollaries apply to 
columns as well as to rows. Our second rather technical remark is based on 
the material in Section 1.5. Note that our treatment of determinants has made 
no reference to the field of scalars with which we have been working. In 
particular, in proving Corollary 2 of Theorem 4.3, what we actually showed 
was that det A = -det A, and hence 2det A = 0. But if ^ happens to be of 
characteristic 2, then we can not conclude from this that det A = 0. However, 
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in this case it is possible to prove the corollary directly through use of the 
expansion by minors to be discussed in Section 4.3 (see Exercise 4.3.19). This 
is why we remarked earlier that we assume our fields are not of characteristic 
2. In fact, for most applications, the reader could just as well assume that we 
are working over either the real or complex number fields exclusively. 

4.2 ADDITIONAL PROPERTIES OF DETERMINANTS 

In this section we present a number of basic properties of determinants that 
will be used frequently in much of our later work. In addition, we will prove 
that three fundamental properties possessed by any determinant are in fact 
sufficient to uniquely define any function that happens to have these same 
three properties (see Theorem 4.9 below). 

Theorem 4.4 Suppose A G MniSF) and let B G MnC?0 be row equivalent to 
A. 

(a) If B results from the interchange of two rows of A, then det B = 
-det A. 

(b) If B results from multiplying any row (or column) of A by a scalar k, 
then det B = k det A. 

(c) If B results from adding a multiple of one row of A to another row, 
then det B = det A. 

Proof By Corollary 1 of Theorem 4.3, a type a elementary row transforma- 
tion merely changes the sign of det A. Next, Theorem 4.2 shows that a type |3 
transformation multiplies det A by a nonzero scalar (choose r = constant and 
s = in the statement of the theorem). Now consider a type y transformation 
that adds k times row j to row i. Then Bj = Aj + kAj so that applying Theorem 
4.2 and Corollary 2 of Theorem 4.3 we have 

detfi = det(fii,...,5,.,...,5^.,...,fij 

= det(Ai, ... , A^ + kAj, ... , Aj, ... , A„) 
= det(Ai, ... , A^, ... , Aj, ... , AJ 

+A:det(Ai, ... , Aj, ... , Aj, ... , A„) 

= detA + k-0 
= detA . I 

Corollary If R is the reduced row-echelon form of a matrix A, then det R = 
if and only if det A = 0. 

Proof This follows from Theorem 4.4 since A and R are row-equivalent. I 
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A matrix A G Mn(^ is said to be upper-triangular if ajj = for i > j. 
Similarly, A is said to be lower-triangular if ajj = for i < j. In other words, an 
upper-triangular matrix has all zeros below the main diagonal, and a lower- 
triangular matrix has all zeros above the main diagonal. 

Theorem 4.5 If A G MnCF) is a triangular matrix, then det A = n?=i 

Proof If A is lower-triangular, then A is of the form 



«11 








• 


•• 




<^22 





• 


•• 


«31 


a^2 


^^33 


• 


•• 



"n2 "n3 "n4 "nn/ 

Since det A = Sc^Sn (sgn a)ai ai • • • ^nan , we claim that the only nonzero 
term in the sum occurs when o is equal to the identity permutation. To see 
this, consider the nonzero term t = ai oi • • • % on for some a G Sn. Since a^ = 
for i < j, we must have al = 1 or else ai oi = = t. Now consider a2 a2 • 
Since al = 1 we must have o2^ I, and hence the fact that a2 a2 = for 2 <o2 
means that only the o2 = 2 term will be nonzero. Next, since al = 1 and o2 = 
2, we must have a3 1 or 2 so that as 03 = for 3 < a3 means that only as 3 
can contribute. Continuing in this manner, we see that only the term t = a^ • • • 
ann is nonzero, and hence 

n 

detA = aii ••• a„„ =]^a,, . 

1=1 

If A is an upper-triangular matrix, then the theorem follows from Theorem 
4.1. I 

Corollary If A = (ay) is diagonal, then det A = Hiaii . 

It is an obvious corollary of this theorem that det I = 1 as we mentioned 
before. Another extremely important result is the following. 

Theorem 4.6 A matrix A G MnCF) is singular if and only if det A = 0. 

Proof Let R be the reduced row-echelon form of A. If A is singular, then 
r(A) < n so that by Theorem 3.9 there must be at least one zero row in the 
matrix R. Hence det R = by Theorem 4.2, Corollary 2, and therefore det A = 
by the corollary to Theorem 4.4. 
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Conversely, assume that r(A) = n. Then, by Theorem 3.10, we must have 
R = In so that det R = 1. Hence det A by the corollary to Theorem 4.4. In 
other words, if det A = it follows that r(A) < n. ■ 

We now prove that the determinant of a product of matrices is equal to the 
product of the determinants. Because of the importance of this result, we will 
present two different proofs. The first is based on the next theorem. 

Theorem 4.7 If E E MnCTO is an elementary matrix and A E MnClTO, then 

det(EA) = (detE)(detA) . 

Proof Recall from Theorem 3.22 that e(A) = e(I)A = EA. First note that if e 
is of type a, then det E = -det 1 = -1 (Theorem 4.3, Corollary 1), and 
similarly det e(A) = -det A. Hence in this case we have 

det(EA) = dete(A) = (-l)detA = (detE)(detA) . 

If e is of type |3, then using Theorem 4.2 we have det E = det e(I) = k det I = k 
so that 

det(EA) = dete(A) = kdetA = (detE)(detA) . 

Finally, if e is of type y, then Theorem 4.5 shows us that det E = det e(I) = 1 
and hence 

det(£'A) = dete(A) 

= det A (see the proof of Theorem 4.4) 
=(detE)(detA) . 

This proves the theorem for each of the three types of elementary row opera- 
tions. I 

Theorem 4.8 Suppose A, B e Mn(^. Then det(AB) = (det A)(det B). 

Proof 1 If either A or B is singular, then so is AB (Corollary to Theorem 
3.20). Hence (by Theorem 4.6) it follows that either det A = or det B = 0, 
and also det(AB) = 0. Therefore the theorem is true in this case. 

Now assume that A and B are both nonsingular. From Theorem 3.24 we 
may write A = Ei • • • Er so that repeated application of Theorem 4.7 yields 



4.2 ADDITIONAL PROPERTIES OF DETERMINANTS 



179 



detAB = det(E^---E^B) 

= detEidet(E2---E^B) 

= det^i det^j det(Ej ■ • E^B) 

det£:i---det£:,detfi 

= det £1 • • • det E^_2 det(£:,_i det E^ ) det B 

det(£:i •••£:,) det 5 

= (detA)(detfi) . I 

Proof 2 If C = AB, then Q = (AB), = 2j a.jBj for each i = 1, . . . , n (see 
Section 3.6). From Corollary 1 of Theorem 4.2 we then have 

detC = det(Ci, ... , C„) 

Now, according to Corollary 2 of Theorem 4.3 we must have jk ?^ jm (for k ?£ 
m) so that we need consider only those terms in this expression for det C in 
which (ji, . . . , jn) is a permutation of (1, . . . , n). Therefore 

detC = ai • • •«« an det(5^i, ...,B^J 

and hence by Theorem 4.3 we have 

detC = 2^e5 ai ^i---a„ ^„(sgn c7)det(5i, ... , BJ 

= (det 5)2^^5 (Sgn o)a^ al'-'^nan 

= (det5)(detA) . I 

Corollary If A e MnC!F) is nonsingular, then det A"' = (det A)"'. 

Proof If A is nonsingular, then A"' exists by Theorem 3.21 so that AA"' = I. 
Hence applying Theorem 4.8 shows that 

1 = detl = det(AA-0 = (det A) (det A"') . 

This implies 

detA-' = (det A)-' . I 



We now show that three of the properties possessed by any determinant 
are in fact sufficient to uniquely define the determinant as a function D: 
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MnCn ~^ !F- By way of terminology, a function D: MnClF) ^ ^ is said to be 
multilinear if it is linear in each of its components. In other words, if D(A) = 
D(Ai, . . . , An) and Aj = Bj + Ci for any i = 1, . . . , n then 

D(A) = D(A„ . . . , Ai-i, B, + C„ Ai+i , . . . , An) 
= D(Ai, . . . , Ai_i, Bi, Ai+i , . . . , An) 

+ D(Ai, . . . , Aj-i, Ci, Aj+i , . . . , An) 

and if Ai = kBi for any k e ^, then 

D(A) = D(Ai, . . . , kBi, . . . , An) = kD(Ai, . . . , Bi, . . . , An) . 

Note Theorem 4.2 shows that our function det A is multilinear. 

Next, we say that D: MnCD ^ ^ is alternating if D(A) = whenever A 
has two identical rows. From Corollary 2 of Theorem 4.3 we see that det A is 
alternating. To see the reason for the word alternating, suppose that Ai = Aj = 
Bi + Ci and D is both multilinear and alternating. Then 

= D(A) = D(A„ . . . , Ai, . . . , Aj, . . . , An) 

= D(Ai, . . . , Bi + Ci, . . . , Bi + Ci, . . . , An) 

= D(Ai, . . . , Bi, . . . , Bi + Ci, . . . , An) 

+ D(A„ . . . , Ci, . . . , Bi + Ci, . . . , An) 

= D(Ai, . . . , Bi, . . . , Bi,. . . . , An) 

+ D(A„ . . . , Bi, . . . , Ci, . . . , An) 
+ D(A„ . . . , Ci, . . . , Bi, . . . , An) 
+ D(Ai, . . . , Ci, . . . , Ci, . . . , An) 

= + D(Ai, . . . , Bi, . . . , Ci, . . . , An) 

+ D(Ai, . . . , Ci, . . . , Bi, . . . , An) + 

so that 

D(Ai, . . . , B, . . . , C, . . . , An) = -D(Ai, . . . , C, . . . , B, . . . , An) . 

Thus, to say that D is alternating means that D(A) changes sign if two rows of 
A are interchanged. 

Finally, let {Ei} be the n row vectors of In (note that Ej, . . . , En form the 
standard basis for f^). Then, as we saw in Theorem 4.5, 

det(Ei, . . . , En) = det I = 1 . 

If we consider a permutation a £ Sn, then from Theorem 4.3 we see that 



4.2 ADDITIONAL PROPERTIES OF DETERMINANTS 



181 



det(Eoi , . . . , Eon) = (sgn a)det(Ei, . . . , En) = sgn a . 

We are now in a position to prove the uniqueness of the determinant func- 
tion. 

Theorem 4.9 Let D: MnCH ^ .IF be a multilinear and alternating function 
with the additional property that D(I) = 1. If D: MnClF) ^ ^ is any other func- 
tion with these properties, then D = D. In particular, the determinant is the 
only such function. 

Proof It follows from the above discussion that our function det has all three 
of the properties given in the theorem, and hence we must show that it is the 
only such function. Let Ai, . . . , An be any set of n vectors in and define 
the function A: J° x • • • x J° by 

A(Ai, . . . , An) = D(Ai, . . . , An) - D(Ai, . . . , An) . 

We must show that A(A,, . . . , An) = 0. 

It should be clear that A is multilinear and alternating, but that 

A(I) = A(Ei, . . . , En) = D(I) - D(I) = . 

Since {Ej} is the standard basis it follows that for any Aj £ we have 
Ai = Sj Cij Ej for some set of scalars Cij. Using this and the properties of A, we 
then have 

A(A,-,A,) = A(2,q,,£,,...,2,.c^,.£,.) 

At this point, each is summed from 1 to n. However, A is alternating so that 
A(Ej, , . . . , Ej„) = if jk = jm for any k, m = 1, . . . , n. Therefore the nonzero 
terms occur only when (ji, . . . , jn) is some permutation of (1, ... , n) and 
hence 

A(Ai, . . . , An) = 2oeS„ Ci ol • • • Cn on A(Eoi , . . . , Eon) • 

Since D and D are alternating, we have 

D(Eoi , . . . , Eon) = (sgn a) D(Ei, . . . , En) = (sgn a) D(I) = sgn a 
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and similarly for D (note that this follows from Theorem 1.2). Therefore we 
find that 

A(Eol , . . . , Eon) = D(Eal , . . . , Ean) - D(Eol , . . . , Eon) 

= sgn a - sgn a = 

and hence A(Ai, . . . , An) = 0. ■ 

Suppose that D is a multilinear and alternating function on the set of all n- 
square matrices over 'J. (Here we do not require that D(I) =1.) If we write the 
rows of a matrix A G MnCiT) as Ai, . . . , An then 

n 

.7 = 1 

where {Ej} are the rows of the identity matrix In. Exactly as we did in the pre- 
ceding proof for the function A, we may write 

D(A) = D(A„...,An) 

= D(2ji aijiEj, , . . . , anj„Ej J 

= 2j,...j„aij, •••anj„D(Ej, ,...,Ej„) 

= 2aeSn ai al • • • an an D(Eal , • • • , Ean) 

= 2aGS„ (sgn a) ai ai • • • an an D(Ei, . . . , En) 

= (det A) D(I) . 

Note that this is actually a quite general formula, and says that any multilinear 

and alternating function D defined on A G MnClT) is just the determinant of A 
times D(I). We will use this formula later in the chapter to give a simple proof 
of the formula for the determinant of a block triangular matrix. 



Exercises 

1. Compute the determinants of the following matrices directly from the 
definition: 



(\ 



{a) 



4-2 3 
2 5-1 



(20 1\ 
3 2-3 
-1 -3 5 
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2. Consider the following real matrix: 



A = 



(1 


1 


9 


1 


4 


3 


-1 


2 


1 


4 


3 


2 


.3 


2 


1 


4 



Evaluate det A by reducing A to upper-triangular form and using 
Theorem 4.4. 

3. Using the definition, show that 





^1 ^2 ^3 h 



^3 ^4 



= a, 



d'j d'l cIa 



4. Evaluate the determinant of the following matrix: 



(0 




1 



1\ 

1 





5. If A is nonsingular and A ^ = A^, show that det A = +1 (such a matrix A 
is said to be orthogonal). 



Consider a complex matrix U G Mn(C). 

(a) If U* = (u,j*), show that det U* = (det U)*. 

(b) Let = U*^ (this is called the adjoint or conjugate transpose of 
U, and is not to be confused with the classical adjoint introduced in the 
next section). Suppose U is such that U^U = UU''" = I (such a U is said to 
be unitary). Show that we may write detU = e'* for some real (|). 



7. 



If A is an n X n matrix and k is a scalar, show that: 

(a) det(kA) = k" det A using Theorem 4.4(b). 

(b) det A" = (det A)". 
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8. (a) If A is a real n x n matrix and k is a positive odd integer, show that 
A'^ = In implies that det A = 1. 

(b) If A° = for some positive integer n, show that det A = 0. (A matrix 
for which A° = is said to be nilpotent.) 

9. If the anticommutator [A, B] + = AB + BA = 0, show that A and/or B in 
MnCF) must be singular if n is odd. What can you say if n is even? 

10. Suppose C is a 3 X 3 matrix that can be expressed as the product of a 3 x 
2 matrix A and a 2 x 3 matrix B. Show that det C = 0. Generalize this 
result to n X n matrices. 

11. Recall that A is symmetric if A^ = A. If A is symmetric, show that 

det(A + B) = det(A + BT) . 

12. Recall that a matrix A = (aij) is said to be antisymmetric if A^ = -A, i.e., 
a^ij = -aji . If A is an antisymmetric square matrix of odd size, prove that 
det A = 0. 

13. (a) Recall (see Exercise 3.6.7) that if A G Mn(J), then Tr A = 2, a,, . If 
A is a 2 X 2 matrix, prove that det(l + A) = 1 + det A if and only if Tr A = 
0. Is this true for any size square matrix? 

(b) If layl « 1, show det(I + A) s 1 + Tr A. 

14. Two matrices A and A' are said to be similar if there exists a nonsingular 
matrix P such that A' = PAP"' . The operation of transforming A into A' in 
this manner is called a similarity transformation. 

(a) Show that this defines an equivalence relation on the set of all matri- 
ces. 

(b) Show that the determinant is invariant under a similarity transforma- 
tion. 

(c) Show that the trace (Exercise 3.6.7) is also invariant. 

15. Consider the matrices 





(2 





-1^ 




( 3 


2 


-4\ 


A = 


3 





2 


B = 


1 





-2 




.4 


-3 


I 




r2 


3 


3. 



(a) Evaluate det A and det B. 
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(b) FindABandBA. 

(c) Evaluate det AB and det B A. 

16. Show that 





+ XGj 


q + yb^ + za^ 




aj 


^1 


Cl 




£»2 + 


Ci + ybi + za^ 






^2 


Cl 


«3 


&3 + xa^ 


C3 + yb^ + za3 




«3 


^3 


C3 



17. Find all values of x for which each of the following determinants is zero: 





x-1 1 1 




1 


X 


X 


(a) 


x-4 1 


(b) 


X 


1 


X 




x-1 




X 


X 


1 





1 


X 


x^ 


(c) 


1 


2 


4 




1 


3 


9 



18. Show that 

(a) det(Ai + A2, A2 + A3, A3 + AO = 2 det(Ai, A2, A3). 

(b) det(Ai + A2, A2 + A3, A3 + A4, A4 + Ai) = 0. 

19. Given a matrix A, the matrix that remains after any rows and/or columns 
of A have been deleted is called a submatrix of A, and the determinant 
of a square submatrix is called a subdeterminant. Show that the rank of 
a matrix A is the size of the largest nonvanishing subdeterminant of A. 
[Hint: Think about Theorem 3.9, Corollary 2 of Theorem 4.2, and 
Theorem 4.4.] 

20. Show that the following determinant is zero: 

(a + iy (a + iy (a + 3)2 
b^ (b + iy {b + lf ib + 3f 

C2 (C + 1)2 (c + 2)2 (c+3)2 

J2 {d + \f {d + lf (J + 3)2 



[Hint: You need not actually evaluate it.] 
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21. Show that 



1 


6 


11 


16 


21 


2 


7 


12 


17 


22 


3 


8 


13 


18 


23 


4 


9 


14 


19 


24 


5 


10 


15 


20 


25 



22. (a) If E is an elementary matrix, show (without using Theorem 4.1) that 
det E'^ = det E. 

(b) Use Theorem 3.24 to show that det A'^ = det A for any A E Mn(J). 

23. Use the material of this section to give a proof (independent of Chapter 3) 
that the product of nonsingular matrices is nonsingular. 



4.3 EXPANSION BY MINORS 

We now turn our attention to methods of evaluating determinants. Since Sn 
contains n! elements, it is obvious that using our definition of det A becomes 
quite impractical for any n much larger than four. Before proceeding with the 
general theory of minors, let us first present a method of evaluating determi- 
nants that is based on Theorem 4.5. All we have to do is reduce the matrix to 
triangular form, being careful to keep track of each elementary row operation 
along the way, and use Theorem 4.4. One example should suffice to illustrate 
the procedure. 

Example 4.2 Consider the matrix A given by 



Then we have 



A = 



detA = 



= 2 



-1 3\ 




2 -1 




2^ 




2 -1 


3 


1 2 - 


1 


-3 . 


2 


1 -1/2 


3/2 


1 2 


-1 


-3 


2 



(1/2)A 
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1 


-1/2 


3/2 


= 2 





5/2 


-5/2 







-3/2 


13/2 




1 


-1/2 


3/2 


= 2 





5/2 


-5/2 










5 



-A1+A2 

3Aj + A3 



■(3/5)A2+A3 

= (2)(l)(5/2)(5) = 25 . 

The reader should verify this result by the direct calculation of det A. / 

We now begin our discussion of the expansion by minors. Suppose A = 
(ay) £ MjiCF), and note that for any term ar or in the definition of det A and 
for any s = 1, . . . , n we may factor out all terms with or = s and then sum 
over all s. This allows us to write 



det A = 2 (sgna)ai 



o\ "' ar " ' an 



a&S„ 
n 



= 2^" 2 (sgna)ai 

s=l <7G5„, ar=s 
n 



ol "' ^r-l o(r-\)^r+\ a(r+\) 



where 2oeSn, or = s means to sum over all a £ Sn subject to the condition that 
or = s, and 



^ri ~ 2 (^S'^^M al '" ^r-\ a(,r-l)^r+l a{r+l) " ' an • 
a&S„, ar=s 



The term a'rs is called the cofactor of ars . Since the sum is over all a G Sn 
subject to the condition or = s, we see that a'rs contains (n - 1)! terms. Indeed, 
it should be apparent that a'rs looks very much like the determinant of some 
(n - 1) X (n - 1) matrix. 

To see that this is in fact the case, we define the matrix Aj-s G Mn-iCT) to 
be the matrix obtained from A G Mn(i70 by deleting the xth row and s,th col- 
umn of A. The matrix Ars is called the rs?/i minor matrix of A, and det Ars is 
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called the rsth minor of A. We now prove the method of calculating det A 
known as the "expansion by minors." 

Theorem 4.10 Suppose A = (ay) E MnCO- Then for any r = 1, . . . , n we 

have 

n 

det A = ^a^Xs 

where 

«; =(-!)'■- det A„ . 

Proof We saw above that det A = 2s°= larsa'rs where each a'rs depends only 
on those elements of A that are not in the rth row or the sth column. In partic- 
ular, consider the expansion det A = ana'n + • • • + ama'm and look at the 
coefficient a'u of an. By definition, we have 

CTes„, CTi=i 

where each term in the sum is a product of elements, one from each row and 
one from each column of A except for the first row and column, and the sum 
is over all possible permutations of the remaining n - 1 columns. But this is 
precisely the definition of det A^, and hence a'u = det A^. (Remember that 
the rth row of An is (An)r = (ar+i 2 , • • • , ar+i n)-) 

We now need to find the coefficient a'rs for any ars- To do this, we start 
with the matrix A, and then move the rth row up to be the new first row, and 
move the sth column left to be the new first column. This defines a new 
matrix B such that b,, = ars and Bn = Ars (note this implies that detB,i = 
det Ars). Moving the rth row involves r - 1 interchanges, and moving the sth 
column involves s - 1 interchanges. Hence applying Corollary 1 of Theorem 
4.3 to both rows and columns, we see that 



detB = (-l)'+'-2 det A = (-l)'+MetA . 



If we expand det B by the first row and expand det A by the rth row, we 
find 

2; =det5 = (-l)-MetA = (-l)-2;«X • 

Now remember that the set {bu, . . . , b,n} is just the set {ari , . . . , am} taken 
in a different order where, in particular, bn = ars. Since the rth row of A is 
arbitrary, we may assume that a^ = for all j 9^ s. In this case, we have 
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det B = biib'ii = (-If^' det A = (-1)'+' ars a'rs 
where bn = ars, and therefore 

b'li = (-l)"^a'rs 

or 

a'rs = (-l)"^b'n . 

At the beginning of this proof we showed that a'n = det A^, and hence an 
identical argument shows that b'n = det Bn. Putting all of this together then 
results in 

a'rs = (-ir^b'n = (-l)"MetBn = (-l)"MetArs . I 

The reader may find it helpful to repeat this proof by moving the rth row 
down and the sth column to the right so that bnn = ars- In this case, instead of 
an we consider 

Kn = 2 ^^^^ ^^^1 
aeS„ , an=n 

which is just det Ann since Ann £ Mn-i(iF) and the sum over all o G Sn 
subject to an = n is just the sum over all a G Sn-i . It then follows again that 

b'nn = det Bnn = det Ars and b'nn = (-1)"' a'rs. 

Corollary 1 Using the same notation as in Theorem 4.10, for any s = 1, . . . , 
n we have 

n 

detA = 2)a.X. • 

(Note that here det A is expanded by the sth column, whereas in Theorem 4.10 
det A was expanded by the rth row.) 

Proof This follows by applying Theorem 4. 10 to A^ and then using Theorem 
4.1. I 

Theorem 4.10 is called expansion by minors of the rth row, and 
Corollary 1 is called expansion by minors of the sth column. (See also 
Exercise 11.3.9.) 

Corollary 2 Using the same notation as in Theorem 4. 10, we have 
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s=l 
n 



Proof Given A = (aij) e MnCH, we define B e MnCF) by B; = A; for i r and 
Br = Ak where r ^ k. In other words, we replace the nth row of A by the kth 
row of A to obtain B. Since B has two identical rows, it follows that det B = 0. 
Noting that Brs = Ars (since both minor matrices delete the rth row), we see 
that b'rs = a'rs for each s = 1, . . . , n. We therefore have 

n n n 

= det 5 = 2 KKs = 2 = 2 ^-^^^^^ • 

.V = l .5 = 1 S = l 

Similarly, the other result follows by replacing the sth column of A by the 
kth column so that brs = ^rk , and then using Corollary 1. I 



Example 4.3 Consider the matrix A given by 





(2 


-1 


5\ 


A = 





3 


4 




.1 


2 


-3. 



To illustrate the terminology, note that the (2, 3) minor matrix is given by 



^23 ~ 



^2 -n 

A 2 



and hence the (2, 3) minor of A is det A23 = 4 - (-1) = 5, while the (2, 3) 
cofactor of A is (-1)^+^ det A23 = -5. 

We now wish to evaluate det A. Expanding by the second row we have 



det A — Cl2i<^2\ ^22^22 "I" ^23*^23 



= + (-l)n3) 
= 3(-6-5)-4(4 + l) = -53 



2 5 




2 -1 


1 -3 


+ (-1)5(4) 


1 2 



The reader should repeat this using other rows and columns to see that they all 
yield the same result. / 
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Example 4.4 Let us evaluate det A where A is given by 



5 


4 


2 


1 


2 


3 


1 


-2 


5 


-7 


-3 


9 


1 


-2 


-1 


4 



In view of Theorem 4.4, we first perform the following elementary row opera- 
tions on A: (j) Ai Ai - 2A2, (//) A3 A3 + 3A2, (///) A4 ^ A4 + Aj. This 
results in the following matrix B: 



(1 


-2 





5^ 


2 


3 


1 


-2 


1 


2 





3 


.3 


1 





2. 



Since these were all type y operations it follows that det B = det A, and hence 
expanding by minors of the third column yields only the single term 



detA = (-1)2+3 



1 -2 5 
1 2 3 
3 1 2 



This is easily evaluated either directly or by reducing to a sum of three 2x2 
determinants. In any case, the result is det A = 38. / 

We are now in a position to prove a general formula for the inverse of a 
matrix. Combining Theorem 4.10 and its corollaries, we obtain (for k, r, s = 1, 
. . . , n) 

n 

=^^detA (la) 

n 

=^fadetA (lb) 

Since each a'ij G J, we may use the them to form a new matrix (a'y) E MnClTO. 
The transpose of this new matrix is called the adjoint of A (or sometimes the 
classical adjoint to distinguish it from another tj^e of adjoint to be discussed 
later) and is denoted by adj A. In other words. 



adj A = (a'y)T 
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Noting that lij = 8ij, it is now easy to prove the following. 

Theorem 4.11 For any A G Mn(^ we have A(adj A) = (det A)l = (adj A) A. 
In particular, if A is nonsingular, then 

^-1 = adj A 
det A 

Proof Using (adj A)sr = a'rs, we may write equation (la) in matrix form as 

A(adj A) = (det A)I 

and equation (lb) as 

(adj A)A = (det A)I . 
Therefore, if A is nonsingular then det (Theorem 4.6), and hence 

A(adj A)/det A = I = (adj A)A/det A . 
Thus the uniqueness of the inverse (Theorem 3.21, Corollary 1) implies that 

A-' = (adj A)/det A . I 
It is important to realize that the equations 

A(adjA) = (detA)I 

and 

(adj A)A = (det A)I 

are valid even if A is singular. We will use this fact in Chapter 8 when we pre- 
sent a very simple proof of the Cayley-Hamilton Theorem. 

Example 4.5 Let us use this method to find the inverse of the matrix 





^-1 


2 


n 


A = 





3 


-2 




. 2 


-1 


0. 



used in Example 3.11. Leaving the details to the reader, we evaluate the co- 
factors using the formula a'rs = (-1)'"^^ det Ars to obtain a'u = -2, a'12 = -4, 
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a'i3 = -6, a'21 = -1, a'22 = -2, a'23 = 3, a'31 = -7, a'32 = -2, and a'33 = -3. 
Hence we find 

1-1 -1 -7^ 



adjA = 



-4 -2 -2 
-6 3 -3 



To evaluate det A, we may either calculate directly or by minors to obtain 
det A = -12. Alternatively, from equation (la) we have 





^1 


2 


1^ 




/ 


-2 


-1 


-1\ 


(detA)/ = A(adjA) = 





3 


-2 






-4 


-2 


-2 




. 2 - 


-1 


Oy 




\ 


-6 


3 






-12 


0] 






^1 





0] 






-12 







-12 





1 












-12 






.0 





\ 





so that we again find that det A = -12. In any case, we see that 





'1/6 


1/12 


7/12^ 




1/3 


1/6 


1/6 




I1/2 


-1/4 


1/4 j 



which agrees with Example 3. 1 1 as it should. / 

If the reader thinks about Theorem 3.9, Corollary 2 of Theorem 4.2, and 
Theorem 4.4 (or has already worked Exercise 4.2.19), our next theorem 
should come as no real surprise. By way of more terminology, given a matrix 
A, the matrix that remains after any rows and/or columns have been deleted is 
called a submatrix of A. (A more precise definition is given in Section 4.6.) 



Theorem 4.12 Let A be a matrix in MmxnC^O, and let k be the largest integer 
such that some submatrix B £ Mk(^ of A has a nonzero determinant. Then 
r(A) = k. 

Proof Since B is a k x k submatrix of A with det ^ follows from 

Theorem 4.6 that B is nonsingular and hence r(B) = k. This means that the k 
rows of B are linearly independent, and hence the k rows of A that contain the 
rows of B must also be linearly independent. Therefore r(A) = rr(A) > k. By 
definition of k, there can be no r x r submatrix of A with nonzero determinant 
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if r > k. We will now show that if r(A) = r, then there necessarily exists an r x 
r submatrix with nonzero determinant. This will prove that r(A) = k. 

If r(A) = r, let A' be the matrix with r linearly independent rows Aj, , . . . , 
Ajj.. Clearly r(A') = r also. But by definition of rank, we can also choose r 
linearly independent columns of A'. This results in a nonsingular matrix A" of 
size r, and hence det A" ;6 by Theorem 4.6. I 

Exercises 

1. Verify the result of Example 4.2 by direct calculation. 

2. Verify the result of Example 4.4. 

3. Verify the terms a'ij in Example 4.5. 

4. Evaluate the following determinants by expanding by minors of either 
rows or columns: 

















2 


5 


5 


3 








2 


-1 


5 


































7 


-8 


2 


3 






(a) 





3 


4 






(b) 


1 


-1 




















4 


-2 








1 


2 


-3 


































-3 


9 


-1 


3 




















3 


1 





4 


2 


1 




3 


2 


2 


3 




2 





1 





5 


1 




1 


-4 


2 


1 







4 


-1 


1 


-1 


2 


(c) 












(d) 














4 


5 


-1 














2 





1 




-1 


-4 


2 


7 
















1 


-1 

























1 





1 



5. Let A G Mn(!f) be a matrix with O's down the main diagonal and 1 's else- 
where. Show that det A = n - 1 if n is odd, and det A = 1 - n if n is even. 

6. (a) Show that the determinant of the matrix 

'1 a a^' 

1 b b^ 

1 c c\ 

is (c - a)(c - b)(b - a). 
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(b) Consider the matrix Vn £ MnCH defined by 
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1 JC^ 

1 X2 X2 



n-\\ 



n-1 



1 -^n -^n^ 



'n 



n-1 



Prove that 



detV„=f](^,-^,) 



where the product is over all pairs i and j satisfying 1 < i, j < n. This 
matrix is called the Vandermonde matrix of order n. [Hint: This should 
be done by induction on n. The idea is to show that 

det Vn = (X2 - Xi)(X3 - Xi) • • • (Xn-1 - Xi)(Xn - Xi) det Vn-1 . 

Perform elementary column operations on Vn to obtain a new matrix V'n 
with a 1 in the (1, 1) position and O's in every other position of the first 
row. Now factor out the appropriate term from each of the other rows.] 

7. The obvious method for deciding if two quadratic polynomials have a 
common root involves the quadratic formula, and hence taking square 
roots. This exercise investigates an alternative "root free" approach. 
(While we will define roots of polynomials in a later chapter, we assume 
that the reader knows that Xq is a root of the polynomial p(x) if and only 
ifp(xo)=0.) 
(a) Show that 



det A 



1 -(X1+X2) 

1 

1 -(3^1+^2) 
1 



XjXj 



-(yi + ji) yji 



(^1 - 3^1 )(^l - J2 )(^2 - yi )(^2 - ^2 ) • 



(b) Using this result, show that the polynomials 



a^x^ + a^x + flj («o ^ 0) 
b^x"^ + b^x + £>2 (^0 ^ 0) 
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have a common root if and only if 



«0 




«2 














K 












bo 




^2 



[//in?: Note that if Xi and X2 are the roots of the first polynomial, then 

(x - Xi)(x - X2) = x^ + (ai/ao)x + az/ao 
and similarly for the second polynomial.] 
8. Show that 



X 








•• 


• 




-1 


X 





•• 


• 







-1 


X 


•• 


• 


a2 











•• 


• -1 





= X" + a„_iX""i + • • • +a, 



Explain why this shows that given any polynomial p(x) of degree n, there 
exists a matrix A £ MnClF) such that det(xl - A) = p(x). (We will discuss 
the matrix A in detail in Chapter 8.) 

9. Consider the following real matrix: 



a 


b 


c 


d\ 


b 


-a 


d 


-c 


c 


-d 


-a 


b 


d 


c 


-b 





Show that det A = implies that a = b = c = d = 0. [Hint: Find AA and 
use Theorems 4.1 and 4.8.] 

10. Consider the usual xy-plane R . Then the two vectors x = (xi, X2) and y = 
(yi, yj) define a quadrilateral with vertices at the points (0, 0), (x,, Xj), 
(y,, y2) and (x, + yi, X2 + y2). Show that (up to a sign) the area of this 
quadrilateral is given by 



4.3 EXPANSION BY MINORS 



197 



3^1 ^2 



[Hint: If the vectors x and y are both rotated by an angle 6 to obtain the 
new vectors x' = (x]', X2') and y' = (yi', y2'), then clearly the area of the 
quadrilateral will remain unchanged. Show (using Example 1.2) it is also 
true that 



f f 




JCy ^2 


yi yi 







and hence you may choose whatever 6 you wish to simplify your calcula- 
tion.] 

What do you think the different signs mean geometrically? We shall 
have quite a bit more to say on this subject in Chapter 11. 

11. Let u, V and w be three vectors in with the standard inner product, and 
consider the determinant G(u, v, w) (the Gramian of {u, v, w}) defined 
by 



G(u, V, w) = 



(u, u) (u, v) (u, 
(v, u) (v, v) (v, w) 
(w, u) (w, v) (w, w) 



Show that G(u, v, w) = if and only if {u, v, w} are linearly dependent. 
As we shall see in Chapter 11, G(u, v, w) represents the volume of the 

parallelepiped in R-^ defined by {u, v, w}.) 
12. Find the inverse (if it exists) of the following matrices: 





'1 


-1 


2^ 






'\ 


3 


2^ 




(a) 


1 


2 









2 


1 


3 






4 


1 










2 


\ 






1-2 


2 






f ^ 




2 


5\ 


(c) 


4 


3 


- 


6 


id) 


-7 




3 


-4 




. 1 


-1 






. 9 




-6 


\ 
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(e) 



(8) 



1 


2 


3 


1 






1 


2 


3 


4 


1 


3 


3 


2 






2 


3 


4 


1 












(/) 










2 


4 


3 


3 




3 


4 


1 


2 


,1 


1 


1 


1, 






,4 


1 


2 


3, 


/2 


3 




2 


4^ 












4 


6 




5 


5 












3 


5 




2 


14 












.2 


2 




-3 


14 













13. Find the inverse of 

'cos0 -sin0 
^-sin0 -COS0 

14. Show that the inverse of the matrix 



G = 







-«3 


-a A 






-(34 


«3 












-«3 







is given by 



15. Suppose that an n-square matrix A is nilpotent (i.e., A'^ = for some 
integer k > 0). Prove that In + A is nonsingular, and find its inverse. 
{Hint: Note that (I + A)(I - A) = I - A^ etc.] 



16. Let P e MnCO be such that P^ = P. If X ^ 1, prove that In - >a' is invert- 
ible, and that 



17. If A = (aij) is a symmetric matrix, show that (a'ij) = (adj A) is also sym- 
metric. 



18. If a, b, c £ IR, find the inverse of 
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' 1 a b' 
-a 1 c . 
-b -c 1^ 

19. Prove that Corollary 2 of Theorem 4.3 is valid over a field of characteris- 
tic 2. [Hint: Use expansion by minors.] 

20. (a) Using A"' = (adj A)/det A, show that the inverse of an upper (lower) 
triangular matrix is upper (lower) triangular. 

(b) IfdL^Q, find the inverse of 



a 


b 


c 


d\ 





a 


b 


c 








a 


b 














21. Let A G Mn(IR) have all integer entries. Show that the following are 
equivalent: 

(a) detA = +l. 

(b) All entries of A"^ are integers. 

22. For each of the following matrices A, find the value(s) of x for which the 
characteristic matrix xl - A is invertible. 





'1 








1 






(a) 


.0 


:i 




{b) 1 




;i 












l\ 





0] 




/O 


1 


2\ 


(c) 








1 


id) 





1 


3 




.0 


1 


0. 




.0 








23. Let A G MjiClF) have exactly one nonzero entry in each row and column. 
Show that A is invertible, and that its inverse is of the same form. 

24. If A e Mn(^, show that det(adj A) = (det A)"-^. 

25. Show that A is nonsingular if and only if adj A is nonsingular. 
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26. Using determinants, find the rank of each of the following matrices: 



(a) 



'I 2 3 4^ 
-1 2 1 0^ 



/-I 1 2 



(b) 



113 
-12 4 1 



4.4 DETERMINANTS AND LINEAR EQUATIONS 

Suppose that we have a system of n equations in n unknowns which we write 
in the usual way as 

n 

^ayXj=bi , i = l,...,n. 

,7 = 1 

We assume that A = (ajj) G Mn(i?0 is nonsingular. In matrix form, this system 
may be written as AX = B as we saw earlier. Since A is nonsingular. A"' 
exists (Theorem 3.21) and det A ^ (Theorem 4.6). Therefore the solution to 
AX = B is given by 

detA 

But adj A = (a'ij)"^ so that 

detA ^detA 

From Corollary 1 of Theorem 4.10, we see that Sibia'y is just the expansion 

by minors of the jth column of the matrix C whose columns are given by = 

A' for i ^ j and O = B. We are thus led to the following result, called 
Cramer's rule. 

Theorem 4.13 If A = (ay) G MnCD is nonsingular, then the system of linear 
equations 

n 

^ayXj=bi, i = l,...,n 

7=1 

has the unique solution 

X,. = ^^det(Ai, ... , Aj-\ B, Aj^\ ... , A") . 
^ detA 
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Proof This theorem was actually proved in the preceding discussion, where 
uniqueness follows from Theorem 3.16. However, it is instructive to give a 



compute using Corollary 1 of Theorem 4.2 and Corollary 2 of Theorem 4.3: 



det(Ai, . . . , A}-\ B, AJ+i, . . . , A") 

= det(Ai, . . . , Aj-\ SA^Xi, Aj+\ . . . , A") 
= 2xi det(Ai, . . . , Aj-\ A\ A^^\ . . . , A") 
= Xj det(Ai, . . . , AJ-i, A\ A^^\ . . . , A") 
= Xj det A . I 



7=1 

has a nontrivial solution if and only if det A = 0. 

Proof We see from Cramer's rule that if det A^Q, then the solution of the 
homogeneous system is just the zero vector (by Corollary 2 of Theorem 4.2 as 
applied to columns instead of rows). This shows that the if the system has a 
nontrivial solution, then det A = 0. 

On the other hand, if det A = then the columns of A must be linearly 
dependent (Theorem 4.6). But the system SjajjXj = may be written as 
SjAJ Xj = where AJ is the ith column of A. Hence the linear dependence of 

the AJ shows that the xj may be chosen such that they are not all zero, and 
therefore a nontrivial solution exists. (We remark that this corollary also fol- 
lows directly from Theorems 3.12 and 4.6.) I 

Example 4.6 Let us solve the system 



more direct proof as follows. We write our system as 2A^ 



Xj = B and simply 



Corollary A homogeneous system of equations 



n 




5x + 2y+ z 
2x- y + 2z 
x+5y- z 



6 



7 



3 



We see that A = (a^) is nonsingular since 



5 



2 



1 



2 - 



1 



2 



= -26;^0 . 



1 



5 - 



1 
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We then have 



JC = 



-1 
26 



3 2 1 
7-12 
6 5-1 



= (-1/26X52) = -2 



y = 



-1 

26 



5 3 1 
2 7 2 
1 6 -1 



= (-l/26)(-78) = 3 



26 



5 2 
2 -1 
1 5 



= (-l/26)(-182) = 7 . / 



Exercises 

1. Using Cramer's rule, find a solution (if it exists) of the following systems 
of equations: 

{a) 3x+ y- z = {b) 2x+ y + lz= 

jc-j + 3z=l 3x-2y+ z= I 

2x + 2y+ z = l -x + 2y + 2z = -l 



(c) 2x-33;+ z = 10 
-x + ?>y + 2z = -2 
Ax + Ay + 5z= 4 

2. By calculating the inverse of 
following systems: 



{d) x + 2y-3z+ t = -9 
2x+ y + 2z- t= 3 
-x+ y + 2z-t=0 
3^ + 43;+ z + 4t= 3 

the matrix of coefficients, solve the 



(a) 2x-3y+ z 
x + 2y + 3z 
3x- y + 2z 



= a 
= b 
= c 



(b) x + 2y + 4z = a 
-x + 3y-2z = b 
2x- y+ z = c 
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(c) 2x+ y + 2z-3t = a 
3x + 2y+3z-5t = b 
2x + 2y+ z- t = c 
5x + 5y + 2z-2t = d 



(d) 6x + y + 4z-3t = a 
2x-y =b 
x+y+ z =c 
-3x -y-2z+ t = d 



3. If det A 9i and AB = AC, show that B = C. 

4. Find, if possible, a 2 x 2 matrix X that satisfies each of the given 
equations: 



(a) 



(2 3\(3 4\ 
12 2 3 



'I 2^ 
.2 h 



(b) 



(0 






'2 1\ 










.1 






,3 2) 



5. Consider the system 



ax + by = a + I3t 
cx + dy = y + dt 



where t is a parameter, p + 8 ?i and 



a b 
c d 



^0 



Show that the set of solutions as t varies is a straight line in the direction 
of the vector 



c dl 



6. Let A, B, C and D be 2 x 2 matrices, and let R and S be vectors (i.e., 2x1 
matrices). Show that the system 

AX + BY = R 
CX + DY = S 



can always be solved for vectors X and Y if 
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«11 


^12 


K 




^21 


^^22 


K 


hi 


Cn 


^12 






^21 


^22 


^21 


^22 



4.5 BLOCK MATRICES 

There is another definition that will greatly facilitate our discussion of matrix 
representations of operators to be presented in Chapter 7. In particular, sup- 
pose that we are given a matrix A = (aij) £ MmxnCT)- Then, by partitioning the 
rows and columns of A in some manner, we obtain what is called a block 
matrix. To illustrate, suppose A £ M3x5(IR) is given by 



A = 



/7 5 5 4 -1^ 
2 1-30 5 
8 2 1 -9 



Then we may partition A into blocks to obtain (for example) the matrix 



where 



A = 



'Ai A2 
V Ai A2 / 



Ai=(V 5 5) 



A2=(4 -1) 





(do not confuse these Ajj with minor matrices). 

If A and B are block matrices that are partitioned into the same number of 
blocks such that each of the corresponding blocks is of the same size, then it is 
clear that (in an obvious notation) 



Mii+^ii 



A + B = 



In addition, if C and D are block matrices such that the number of columns in 
each Cij is equal to the number of rows in each Djk, then the product of C and 
D is also a block matrix CD where (CD)ik = 2j C^ Djk. Thus block matrices 
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are multiplied as if each block were just a single element of each matrix in the 
product. In other words, each (CD)ik is a matrix that is the sum of a product of 
matrices. The proof of this fact is an exercise in matrix multiplication, and is 
left to the reader (see Exercise 4.5. 1). 



Theorem 4.14 If A £ MnClF) is a block triangular matrix of the form 



(An 




n2 



^22 



^13 



^23 







A \ 



^2it 



^kk/ 



where each Aii is a square matrix and the O's are zero matrices of appropriate 
size, then 

k 

detA = []detA, . 



(=1 



Proof What is probably the simplest proof of this theorem is outlined in 
Exercise 4.5.3. However, the proof that follows serves as a good illustration of 
the meaning of the terms in the definition of the determinant. We first note 
that only the diagonal matrices are required to be square matrices. Because 
each Aii is square, we can simply prove the theorem for the case k = 2, and the 
general case will then follow by induction. We thus let A = (aij) £ MnCTO be 
of the form 

(B C\ 

where B = (by) G MrQf), D = (d^) G Ms(^, C = (c^) G M^sCO and r + s = n. 
Note that for 1 < i, j < r we have aij = bjj, for 1 < i, j < s we have ai+r j+r = dy, 
and if i > r and j < r then a^ = 0. From the definition of determinant we have 

det A = (sgna)ai • • • a, „a^^i ^(^^^ • • • an • 

By definition, each a G Sn is just a rearrangement (i.e., permutation) of the n 
elements in Sn. This means that for each o G Sn with the property that oi > r 
for some i < r, there must be some i' > r such that oi' < r. Then for this i' we 
have ai'oi' = 0, and hence each term in det A that contains one of these factors 
is zero. Therefore each nonzero term in the above sum must be over only 
those permutations a such that ai > r if i > r (i.e., the block D), and ai < r if 
i<r (i.e., the block B). 
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To separate the action of the allowed o on the blocks B and D, we define 
the permutations a G Sr and (3 G Ss by ai = ai for 1 < i < r, and (3i = a(i + r) - 
r for 1 < i < s. In other words, each of the allowed a is just some rearrange- 
ment a of the values of i for 1 < i < r along with some rearrangement (3 of the 
values of i for r<i<r + s = n, and there is no mixing between these blocks. 
The permutations a and |3 are thus independent, and since sgn o is defined by 
the number of transpositions involved, this number simply separates into the 
number of transpositions in a plus the number of transpositions in (3. There- 
fore sgn a = (sgn a)(sgn (3). 

The result of this is that all nonzero terms in det A are of the form 



(sgna)Z7i „,(sgn/3)rfi ■■■ 



I3s 



Furthermore, every term of this form is included in the expression for det A, 
and hence det A = (det B)(det D). I 

There is another way to prove this theorem that is based on our earlier for- 
mula 

D(A) = (detA)£)(/) (2) 

where D is any multilinear and alternating function on the set of all n x n 
matrices (see the discussion following the proof of Theorem 4.9). To show 
this, consider the block triangular matrix 



(A B\ 
^0 C 



where A and C are square matrices. Suppose we define the function 



D(A, B, C) = 



A B 
C 



If we consider the matrices A and B to be fixed, then this is clearly a multi- 
linear and alternating function of the rows of C. Applying equation (2), this 
may be written as 

D(A,B,C) = (det C) D(A, B, I) 



where 



D{A, B, I) = 



A B 
/ 
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But using Theorem 4.4(c) it should be obvious that we can subtract suitable 
multiples of the rows of I from the matrix B so that 

D(A, B, I) = D(A, 0, 1) . 

Applying the same reasoning, we observe that D(A, 0, 1) is a multilinear and 
alternating function of the rows of A, and hence (2) again yields 

D(A,0,I) = (det A) D(1, 0, 1) . 

Putting all of this together along with the obvious fact that D(I, 0, I) = 1, we 
obtain 

D{A, B, C) = (det C)Z)(A, B, I) 
= (det C)£)(A, 0, /) 
= (detC)(det A)D{I, 0, /) 
= (detC)(det A) 

which agrees with Theorem 4. 14. 

Example 4.7 Consider the matrix 



1 


-1 


2 


3^ 


2 


2 





2 


4 


1 


-1 


-1 


1 


2 


3 


0. 



By the addition of suitable multiples of one row to another, it is easy to row- 
reduce A to the form 



B = 



1 -1 
4 








2 3 
-4 -4 



-4 


-8 


4 






with det B = det A. Since B is in block triangular form we have 

1 -1 -4 -8 
detA = detfi= =4(32) = 128 . 



1 -1 


-4 


-8 


4 


4 
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Exercises 

1. Prove the multiplication formula given in the text (just prior to Theorem 
4. 14) for block matrices. 

2. Suppose A e M„en, D e MmCD, U e MnxmCD and V e MnxxnCD, and 
consider the (n + m) x (n + m) matrix 

'A U' 



M=\ 



If A ' exists, show that 

\-VA-' lj\v d] \q -VA-'U + d] 



and hence that 



A U 
V D 



= (detA)det(Z)-yA-i[/) 



3. Let A be a block triangular matrix of the form 



A = 



'B C 

,0 d\ 



where B and D are square matrices. Prove det A = (det B)(det D) by using 
elementary row operations on A to create a block triangular matrix 



A = 



'b 



D 



where B and D are upper-triangular. 
4. Show 





3'- 











4.6 THE CAUCHY-BINET THEOREM 

This section deals with a generalization of Theorem 4.8 that is not used any- 
where in this book except in Chapter 8. Because of this, the reader should feel 
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free to skip this section now and come back to it if and when it is needed. Let 
us first point out that while Theorem 4.8 dealt with the product of square 
matrices, it is nevertheless possible to formulate a similar result even in the 
case where A G Mnxp(J) and B G Mpxn(^ (so that the product AB G Mn(^ 
is square and det AB is defined). This result is the main subject of this section, 
and is known as the Cauchy-Binet theorem. 

Before proceeding with our discussion, we briefly present some new nota- 
tion that will simplify some of our formulas significantly, although at first it 
may seem that all we are doing is complicating things needlessly. Suppose 
that we have a matrix A with m rows and n columns. Then we can easily form 
a new matrix B by considering only those elements of A belonging to, say, 
rows 2 and 3 and columns 1, 3 and 4. We shall denote this submatrix B of A 
by B = A[2, 3|1, 3, 4]. What we now wish to do is make this definition 
precise. 

To begin with, let k and n be positive integers with 1 < k < n. We let 
MAP(k, n) denote the set of all mappings from the set k = (1, . . . , k) to the set 
n = (1, . . . , n). For example, if n = 5 and k = 3, we can define a G MAP(3, 5) 
by a(l) = 2, a(2) = 5 and a(3) = 3. Note that an arbitrary a G MAP(k, n) need 
not be injective so that, for example, a' G MAP(3, 5) could be defined by 
a'(l) = 2, a'(2) = 5 and a'(3) = 5. In fact, we will let INC(k, n) denote the set 
of all strictly increasing functions from the set k into the set n. Thus if (3 G 
INC(k, n), then |3(1) < |3(2) < • • • < (3(k). We also denote the mapping |3 by 
simply the k- tuple of numbers (3 = ((3(1), ... , |3(k)). Note that this k- tuple 
consists of k distinct integers in increasing order. 

Now consider the set of all possible permutations of the k integers within 
each k- tuple (3 for every P G INC(k, n). This yields the set INJ(k, n) consisting 
of all injective mappings from the set k into the set n. In other words, if (3 G 
INJ(k, n), then (3 = ((3(1), ... , (3(k)) is a k-tuple of k distinct integers in any 
(i.e., not necessarily increasing) order. In the particular case that k = n, we see 
that the set INJ(n, n) is just the set of all permutations of the integers 1, . . . , n. 
The set INJ(n, n) will be denoted by PER(n). (Note that the set INJ(n, n) is the 
same as the set Sn, but without the additional group structure.) 

Now suppose that A = (a;-) G Mmxn(iF), let a = (i,, . . . , ij G INC(k, m) 
and let (3 = (ji, • • • , jt) ^ INC(t, n). Then the matrix B G Mkxt(^ whose (r, 
s)th entry is aij^ (where 1 < r < k and 1 < s < t) is called the submatrix of A 
lying in rows a and columns (3. We will denote this submatrix by A[a((3]. 
Similarly, we let A(a(|3] G M(ni-k)xtCin denote the submatrix of A whose 
rows are precisely those complementary to a, and whose columns are again 
given by (3. It should be clear that we can analogously define the matrices 
A(a(p) G M(m-k)x(n-t)(^ and A[a((3) G Mkx(n-t)(^. Fortunately, these 
ideas are more difficult to state carefully than they are to understand. 
Hopefully the next example should clarify everything. 
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Example 4.8 Suppose A e MsxeCH, and let a = (1, 3) e INC(2, 5) and p = 
(2, 3, 4) e INC(3, 6). Then 



A[a 1 13] = 
A(a 1 13] = 



«12 


«13 


«14' 






«15 






'^33 


«34; 






«35 


^36/ 














<^26 




'^43 


«44 


A(a 1 /3) = 


«41 




^46 


,^52 


^53 


^54/ 




,«51 


«55 


^56/ 



Before stating and proving the main result of this section, it will be useful 
for us to gather together several elementary computational facts that we will 
need. 



Lemma 4.1 



k / n \ 



ns 



(=1 \7=1 / 



2 n"^'«(') 

aBMAP(k,n)\i=l / 



To see that this is true, we simply look at a particular example. Thus, consider 
the set MAP(2, 3). This consists of all mappings a such that a(i) E 3 for each 
i = 1, 2. It is easy to enumerate all nine possibilities: 

a(l) = 1 and a(2) = 1, 2, or 3 
a(l) = 2 and a(2) = 1, 2, or 3 
a(l) = 3 and a(2) = 1, 2, or 3 

Expanding the product in the natural way we obtain 



/ 3 \ 



'■=1 \7=1 / 



— J^i 1-^21 + + •^11-^23 ■^12-^21 •^12-^22 

+•^12-^23 ■^13-^21 •^13-^22 •^13-^23 



2 -^1 a(l)-^2 a(2) 
aGMA/'(2, 3) 

^ 2 n 



X, 



i a(i) 



a&MAP(2, 3) \ (=1 
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A minutes thought should convince the reader that Lemma 4. 1 is true in gen- 
eral. If it does not, then pick an example and work it out for yourself. 

Recall from Section 4. 1 that we may view the determinant as a function of 
either the rows Aj or columns A^ of a matrix A G Mn(^. Applying the defini- 
tion of determinant and Corollary 2 of Theorem 4.3, we obtain the next fact. 
Note that even though we originally defined A[a|(3] for a, (3 G lNC(m, n), we 
can just as well assume that a, (3 G MAP(m, n). 

Lemma 4.2 If a = (a(l), . . . , a(n)) G MAP(n, n) is not an injection, then for 
any square matrix A = (aij) G MnCT) we have 

n 

detA[«ln]= J (sgn 0)n«a(oe(O = ^ • 

eePER{n) (=1 

Our discussion above showed that the set lNJ(k, n) arose by considering 
all permutations of the k-tuples in lNC(k, n). Let us take a closer look at the 
consequences of this observation. If (a(l), . . . , a(k)) G INJ(k , n), then there 
are n choices for a(l), n - 1 choices for a(2) and so on down to n - (k - 1) 
choices for a(k). In other words, the set INJ(k, n) consists of 

n(n - 1) • • • (n - (k - 1)) = n!/(n - k)! 

mappings. It should also be clear that the set INC(k, n) consists of (g) = 
n!/[k!(n - k)!] mappings since INC(k, n) is just the collection of increasing k- 
tuples taken from the set n. Finally, we recall that PER(k) has k! elements, and 
therefore we see that the number of elements in lNJ(k, n) is just the number of 
elements in lNC(k, n) times the number of elements in PER(k). 

As an example, let n = 4 and k = 3. Then 1NC(3, 4) consists of the 
sequences (1, 2, 3), (1, 2, 4), (1, 3, 4) and (2, 3, 4). If we enumerate all possi- 
ble permutations of each of these, we obtain the following elements of INJ(3, 
4): 



123 


124 


134 


234 


132 


142 


143 


243 


213 


214 


314 


324 


231 


241 


341 


342 


312 


412 


413 


423 


321 


421 


431 


432 



In the next lemma, we let Qa be some quantity that depends on the map- 
ping a, and let a6 denote the composition of a and 6. 

Lemma 4.3 
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■ad 



aBINJ(k, n) a^INC{k, n) e&PER{k) 



A careful look at the above example should make this fact obvious. Note 
that it is not so much a statement about sums as it is about the set INJ(k, n). 

Finally, our last fact is just a restatement of Theorem 4.3 in our current 
notation. 

Lemma 4.4 Let A G MmxnCO where m > n, and suppose that E PER(n) 
and a E MAP(n, m). Then 



We are now in a position to state and prove the main result needed for the 
Cauchy-Binet theorem. Note in the following that the particular case of n = p 
is just Theorem 4.8. 

Theorem 4.15 Suppose A = (ajj) E MnxpCTO and B = (bij) E MpxnCD where 
n < p. Then 



Proof From the definitions of determinant and matrix product we have 



detA[a0|n] = (sgn e)det A[a|n ] . 



det(A5) = 



2 (det A[n I a])(det B{a\n\) 



aeiNC(n, p) 



n 



det(A5)= 2 (sgn0)n(^X 



eePER{n) !=1 



( n P \ 



eePER{n) \i=l j=l J 



Using Lemma 4.1 with Xy = aijbje(i) the right hand side of this expression 
becomes 
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( " ] 



aeAMP(n, p) 



\ i=l l\eE:PER(n) 1=1 



2 n«... 



detB[aln] 



By Lemma 4.2, we need only sum over those a G INJ(n, p). Combining this 
with Lemma 4.3 we obtain for this last expression 



2 2 det5Mlnjfla,„e(o 



i-l I 



Now, applying Lemma 4.4, this becomes 



/ n \ 

2 detfilalnj J (^gn ^g^,) 

aelNCin, p) \eePER(n) i=\ j 



2 (det B[a I « j)(det A[n I a]) 

aE.INC(n, p) 



This completes the proof. I 



Note the particular case of p = n yields an independent proof of Theorem 

4.8. 

The principal result of this section now follows as an easy generalization 
of Theorem 4.15. Thus, suppose that A G MnxsW and B G MsxmCD, and let 
1 < r < min{n, s, m}. If a E INC(r, n) and P G INC(r, m), we have 



and 
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C\a I /3] = 



ARP^ ... ARPr 



Then C[a|(3] = A[a| s] B[s|p] G Mr(J) and we can now apply Theorem 4.15 
to obtain the following corollary, known as the Cauchy-Binet theorem. 

Corollary (Cauchy-Binet) Suppose A E Mnxs(J), B E MsxmW and C = 
AB. Then for any r with 1 < r ^ min{n, s, m} and a G INC(r, n), |3 G 
INC(r, m) we have 

det C[a 1/8]= J idetA[a\(o])idetB[co\ p]) . 

(oeiNC(r, s) 



Example 4.9 Let 



A = 



(I 1 0\ 
1 1 



/-I n 



and B = 



1 

1 



The increasing functions from 2 to 3 are (1, 2), (1, 3) and (2, 3), and hence 



A[2 1 1, 2] = 



\0 h 



A[2 1 1, 3] = 



(I 0\ 
1, 



A[2I2, 3] = 



B[\, 312] = 



1 0\ 

(-1 l\ 

.1 



B[l, 212] = 



S[2, 312] = 



/-I l\ 

,1 oj 



Applying Theorem 4.15 we have 

detAfi = detA[2tl, 2]detB[l, 2 1 2] + detA[2 1 1, 3]detB[l, 3 1 2] 

+detA[2l2, 3]detB[2, 312] 

= (l)(-l) + (l)(-l) + a)(-l) 
= -3 . 



We leave it to the reader to verify that direcdy evaluating the product AB and 
taking its determinant yields the same result. / 



CHAPTER 5 



Linear Transformations 
and IViatrices 

In Section 3.1 we defined matrices by systems of linear equations, and in 
Section 3.6 we showed that the set of all matrices over a field ^ may be 
endowed with certain algebraic properties such as addition and multiplication. 
In this chapter we present another approach to defining matrices, and we will 
see that it also leads to the same algebraic behavior as well as yielding impor- 
tant new properties. 

5.1 LINEAR TRANSFORMATIONS 

Recall that vector space homomorphisms were defined in Section 2.2. We 
now repeat that definition using some new terminology. In particular, a map- 
ping T: U V of two vector spaces over the same field ^ is called a linear 
transformation if it has the following properties for all x, y E U and a E ^: 

(a) T(x + y) = T(x) + T(y) 

(b) T(ax) = aT(x) . 

Letting a = and -1 shows 

T(0) = 

and 



91 c; 
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T(-x) = -T(x) . 

We also see that 

T(x-y) = T(x + (-y)) = T(x) + T(-y) = T(x) - T(y) . 
It should also be clear that by induction we have, for any finite sum, 

T(2aiXi) = 2T(aiXi) = 2aiT(Xi) 
for any vectors Xj G V and scalars ai G ^. 

Example 5.1 Let T: IR^ IR^ be the "projection" mapping defined for any 
u = (x, y, z) G r3 by 

T(u) = T(x, y, z) = (X, y, 0) . 

Then if v = (x', y', z') we have 

T{u + v) = T{x + x', y + y, z + z') 
= (jc + x', j + y, 0) 
= (x, y, 0) + (x', y, 0) 

= r(M)+r(v) 

and 

T(au) = T(ax, ay, az) = (ax, ay, 0) = a(x, y, 0) = aT(u) . 
Hence T is a linear transformation. / 

Example 5.2 Let P G Mn(^ be a fixed invertible matrix. We define a map- 
ping S: MnCF) MnC^ by S(A) = p-'AP. It is easy to see that this defines a 
linear transformation since 

S(aA + B) = p-i(aA + B)P = aP"' AP + p-'BP = aS(A) + S(B) . / 

Example 5.3 Let V be a real inner product space, and let W be any subspace 
of V. By Theorem 2.22 we have V = W W-^, and hence by Theorem 2.12, 
any v G V has a unique decomposition v = x + y where x G W and y G W"^. 
Now define the mapping T: V ^ W by T(v) = x. Then 



and 



T(vi + V2) = X1 + X2 = T(vi)+T(v2) 
T(av) = ax = aT(v) 
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so that T is a linear transformation. This mapping is called the orthogonal 
projection of V onto W. / 

Let T: V W be a linear transformation, and let {ei} be a basis for V. 
Then for any x £ V we have x = 2xiei, and hence 

T(x) = T(2xieO = 2xiT(eO . 

Therefore, if we know all of the T(ei), then we know T(x) for any x G V. In 
other words, a linear transformation is determined by specifying its values on 
a basis. Our first theorem formalizes this fundamental observation. 

Theorem 5.1 Let U and V be finite-dimensional vector spaces over J^, and 
let {ci, . . . , Cn} be a basis for U. If Vi, . . . , Vn are any n arbitrary vectors in V, 
then there exists a unique linear transformation T: U ^ V such that T(ei) = Vi 
for each i = 1, . . . , n. 

Proof For any x G U we have x = 2 1= i Xiei for some unique set of scalars Xj 
(Theorem 2.4, Corollary 2). We define the mapping T by 

n 

(=1 

for any x G U. Since the X; are unique, this mapping is well-defined (see 
Exercise 5.1.1). Noting that for any i = 1, . . . , n we have ei = 2j 8ijej, it fol- 
lows that 

i=l 

We show that T so defined is a linear transformation. 

If X = SxiCi and y = SyiCi, then x + y = 2(Xi + yi)ei, and hence 

T(x + y) = 2(Xi + yi)Vi = SxiV; + SyiVi = T(x) + T(y) . 

Also, if c G ^ then cx = 2(cXi)ei, and thus 

T(cx) = 2(cXi)Vi = c2xiVi = cT(u) 

which shows that T is indeed a linear transformation. 

Now suppose that T': U ^ V is any other linear transformation defined by 
T'(ei) = Vi. Then for any x G U we have 



T'(x) = TXSxiCi) = 2xiT'(ei) = Sx^Vi = 2xiT(ei) = T(2xiei) = T(x) 
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and hence T'(x) = T(x) for all x £ U. This means that T' = T which thus 
proves uniqueness. I 



Example 5.4 Let T G L(J™, J") be a linear transformation from J™ to J", 
and let {Ci, ... , Cm} be the standard basis for We may uniquely define 
T by specifying any m vectors Vi, . . . , Vm in In other words, we define T 
by the requirement T(ei) = Vj for each i = 1, . . . , m. Since T is linear, for any 
X E ^™ we have x = 2T=l XiCi and hence 



Now define the matrix A = (ajj) G Mnxm(91 with column vectors given by 
= Vi G In other words (remember these are columns), 

A' = (aii, . . . , ani) = (Vii, . . . , Vni) = Vi 

where Vi = 2]^= i fjVji and {fj, . . . , fn} is the standard basis for Writing out 
T(x) we have 



Tix) = ^XiVi=Xi 



i=l 



Im 



+ • • • +X„ 



\m m 



and therefore, in terms of the matrix A, our transformation takes the form 



T{x) = 



nm I 



fxA 



We have therefore constructed an explicit matrix representation of the 
transformation T. We shall have much more to say about such matrix repre- 
sentations shortly. / 



Given vector spaces U and V, we claim that the set of all linear transfor- 
mations from U to V can itself be made into a vector space. To accomplish 
this we proceed as follows. If U and V are vector spaces over F and f, g: 
U ^ V are mappings, we naturally define 



and 



(f + g)(x) = f(x) + g(x) 
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(cf)(x) = cf(x) 

for X G U and c G ^. In addition, if h: V — * W (where W is another vector 
space over then we may define the composite mapping h ° g: U W in 
the usual way by 

(hog)(x) = h(g(x)) . 

Theorem 5.2 Let U, V and W be vector spaces over !f, let c G ^ be any 
scalar, and let f, g: U — * V and h: V ^ W be linear transformations. Then the 
mappings f + g, cf, and h ° g are all linear transformations. 

Proof First, we see that for x, y G U and c G ^ we have 

(/ + g)(x + y) = fix + y) + g(x + y) 

= f(x) + f(y) + gix) + g(y) 
= (f + 8)(x) + (f + g)(y) 

and 

(/ + g)(cx) = f(cx) + g(cx) = cfix) + cg(x) = c[f(x) + g{x)] = c{f + g){x) 

and hence f + g is a linear transformation. The proof that cf is a linear 
transformation is left to the reader (Exercise 5.1.3). Finally, we see that 

(h o g)(x + y) = h{g(x + y)) = h(gix) + g(y)) = h(g(x)) + h(giy)) 
= ih°g){x) + ihog)(y) 

and 

(h o g)(cx) = higicx)) = h{cg{x)) = ch{g{x)) = c{h o g){x) 
so that h o g is also a linear transformation. I 

We define the zero mapping 0: U ^ V by Ox = for all x G U. Since 

0(x + y) = = Ox + Oy 

and 

0(cx) = = c(Ox) 

it follows that the zero mapping is a linear transformation. Next, given a map- 
ping f: U V, we define its negative -f: U ^ V by (-f )(x) = -f(x) for all 
X G U. If f is a linear transformation, then -f is also linear because cf is linear 
for any c G ^ and -f = (-l)f (by Theorem 2.1(c)). Lastly, we note that 
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[/ + (-mx) = m + (-f)(x) = fix) + [-fix)] = fix) + fi-x) = fix - X) 

= /(0) = 

for all X G U so that f + (-f ) = (-f ) + f = for all linear transformations f. 

With all of this algebra out of the way, we are now in a position to easily 
prove our claim. 

Theorem 5.3 Let U and V be vector spaces over ^. Then the set of all linear 
transformations of U to V with addition and scalar multiplication defined as 
above is a linear vector space over ^. 

Proof We leave it to the reader to show that the set of all such linear transfor- 
mations obeys the properties (VI) - (V8) given in Section 2.1 (see Exercise 
5.1.4). I 

We denote the vector space defined in Theorem 5.3 by L(U, V). (Some 
authors denote this space by Hom(U, V) since a linear transformation is just a 
vector space homomorphism). The space L(U, V) is often called the space of 
linear transformations (or mappings). In the particular case that U and V 
are finite-dimensional, we have the following important result. 

Theorem 5.4 Let dim U = m and dim V = n. Then 

dim L(U, V) = (dim U)(dim V) = mn . 

Proof We prove the theorem by exhibiting a basis for L(U, V) that contains 
mn elements. Let {ei, . . . , em} be a basis for U, and let { Bi, . . . , Bn} be a 

basis for V. Define the mn linear transformations E'j G L(U, V) by 

E'j(ek) = 

where i, k = 1, . . . , m and j = 1, . . . , n. Theorem 5.1 guarantees that the map- 
pings E^j are unique. To show that {E'j} is a basis, we must show that it is 
linearly independent and spans L(U, V). 
If 

m n 
i=l j=l 

for some set of scalars a^i, then for any e^ we have 



= 2i,jajiE'j(ek) = 2i,jaji6\ej = SjaJkej 
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But the Bj are a basis and hence linearly independent, and thus we must have 
a^k = for every j = 1, . . . , n and k = 1, . . . , m. This shows that the E'j are 
linearly independent. 

Now suppose f £ L(U, V) and let x £ U. Then x = 2iX^ei and 

f(x) = f(2ixieO = 2ixif(eO . 

Since f(ei) G V, we must have f(ei) = 2j c^i Bj for some set of scalars c^i, and 
hence 

f(ei) = SjC-'iCj = 2j,kcjk6Sej = 2]j,k c^kE^'j (eO . 

But this means thatf = 2j,kcjkE''j (Theorem 5.1), and therefore {E'^j} spans 
L(U,V). I 

Suppose we have a linear mapping (j): V -^J^ of a vector space V to the 
field of scalars. By definition, this means that 

(|)(ax + by) = a(|)(x) + b(|)(y) 

for every x, y G V and a, b G ^. The mapping ^ is called a linear functional 
onV. 

Example 5.5 Consider the space MnCT) of n-square matrices over J^. Since 
the trace of any A = (ay) G MnCD is defined by 

(see Exercise 3.6.7), it is easy to show that Tr defines a linear functional on 
M„(^ (Exercise 5.1.5). / 

Example 5.6 Let C[a, b] denote the space of all real-valued continuous func- 
tions defined on the interval [a, b] (see Exercise 2.1.6). We may define a 
linear functional L on C[a, b] by 

for every f G C[a, b]. It is also left to the reader (Exercise 5.1.5) to show that 
this does indeed define a linear functional on C[a, b]. / 

Let V be a vector space over ^. Since ^ is also a vector space over itself, 
we may consider the space L(V, This vector space is the set of all linear 
functionals on V, and is called the dual space of V (or the space of linear 
functionals on V). The dual space is generally denoted by V*. From the proof 
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of Theorem 5.4, we see that if {ei} is a basis for V, then V* has a unique basis 
{coJ} defined by 

o>j(ei) = . 

The basis {coJ} is referred to as the dual basis to the basis {ei}. We also see 
that Theorem 5.4 shows that dim V* = dim V. 

(Let us point out that we make no real distinction between subscripts and 
superscripts. For our purposes, we use whichever is more convenient from a 
notational standpoint. However, in tensor analysis and differential geometry, 
subscripts and superscripts are used precisely to distinguish between a vector 
space and its dual. We shall follow this convention in Chapter 11.) 

Example 5.7 Consider the space V = of all n-tuples of scalars. If we write 
any x G V as a column vector, then V* is just the space of row vectors. This is 
because if (|) G V* we have 

(|)(x) = (|)(2xiei) = 2xi(|)(ei) 

where the e-, are the standard (column) basis vectors for V = Thus, since 
(j)(ei) G ^, we see that every (j)(x) is the product of some scalar (j)(ei) times the 
scalar Xj, summed over i = 1, . . . , n. If we write (j)(ei) = ai, it then follows that 
we may write 



(t)(x) = (t)ixi, ... , x„) = (ai, ... , a„) 



(*) 



or simply (|)(x) = SaiXj. This expression is in fact the origin of the term "linear 
form." 

Since any row vector in can be expressed in terms of the basis vectors 

co^ = (1, 0, . . . , 0), . . . , 0)" = (0, 0, . . . , 1), we see from (*) that the a>i do 
indeed form the basis dual to since they clearly have the property that 

^^(ei) = S^i . In other words, the row vector coJ is just the transpose of the cor- 
responding column vector Cj. / 

Since U* is a vector space, the reader may wonder whether or not we may 
form the space U** = (U*)*. The answer is "yes," and the space U** is called 
the double dual (or second dual) of U. In fact, for finite-dimensional vector 
spaces, it is essentially true that U** = U (in the sense that U and U** are 
isomorphic). However, we prefer to postpone our discussion of these matters 
until a later chapter when we can treat all of this material in the detail that it 
warrants. 
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Exercises 

1 . Verify that the mapping T of Theorem 5. 1 is well-defined. 

2. Repeat Example 5.4, except now let the matrix A = (aij) have row vectors 
Ai = Vi £ What is the matrix representation of the operation T(x)? 

3. Show that cf is a linear transformation in the proof of Theorem 5.2. 

4. Prove Theorem 5.3. 

5. (a) Show that the function Tr defines a linear functional on MnCF) (see 
Example 5.5). 

(b) Show that the mapping L defined in Example 5.6 defines a linear 
functional. 

6. Explain whether or not each of the following mappings f is linear: 

(a) f: IR2 ^ IR defined by f(x, y) = xy. 

(b) f: IR2 ^ R defined by f(x, y, z) = 2x - 3y + 4z. 

(c) f: IR2 defined by f(x, y) = (x + 1, 2y, x + y). 

(d) f: IR3 ^ IR2 defined by f(x, y, z) = (|x|, 0). 

(e) f: IR2 defined by f(x, y) = (x + y, x). 

(f ) f: IR3 ^ IR3 defined by f(x, y, z) = (1, -x, y + z). 

(g) f: IR2 ^ |R2 defined by f(x, y) = (sin x, y). 

(h) f: IR2 ^ IR defined by f(x, y) = |x - y|. 

7. Let T: U — * V be a bijective linear transformation. Define T"' and show 
that it is also a linear transformation. 

8. Let T: U ^ V be a linear transformation, and suppose that we have the 
set of vectors Ui, . . . , Un E U with the property that T(u,), . . . , T(Un) E V 
is linearly independent. Show that {Ui, . . . , Un} is linearly independent. 

9. Let B e Mndf) be arbitrary. Show that the mapping T: MnCO ^ Mn(^ 
defined by T(A) = [A, B] + = AB + BA is linear. Is the same true for the 
mapping T'(A) = [A, B] = AB - BA? 
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10. Let T: -^!F^ be the linear transformation defined by the system 

= -3Xi + ^2 

and let S be the linear transformation defined by the system 

= Xj + X2 

Find a system of equations that defines each of the following linear trans- 
formations: 

(a) 2T (b) T - S (c) 

(d) TS (e) ST (f) T2 + 2S 

11. Does there exist a linear transformation T: IR'^ ^ with the property 
that T(l, -1, 1) = (1, 0) and T(l, 1, 1) = (0, 1)? 

12. Suppose Ui = (1, -1), U2 = (2, -1), U3 = (-3, 2) and Vi = (1, 0), V2 = (0, 1), 
V3 = (1, 1). Does there exist a linear transformation T: IR^ IR^ with the 
property that Tui = Vi for each i = 1,2, and 3? 

13. Find T(x, y, z) if T: ^ R is defined by T(l, 1, 1) = 3, T(0, 1, -2) = 1 
and T(0, 0, 1) = -2. 

14. Let V be the set of all complex numbers considered as a vector space 
over the real field. Find a mapping T: V ^ V that is a linear transforma- 
tion on V, but is not a linear transformation on the space (i.e., the set 
of complex numbers considered as a complex vector space). 

15. If V is finite-dimensional and x,, X2 G V with xi 9^ X2 , prove there exists 
a linear functional f G V* such that f(xi) ^ f(x2). 

5.2 FURTHER PROPERTffiS OF LINEAR TRANSFORMATIONS 

Suppose T G L(U, V) where U and V are finite-dimensional over J^. We 
define the image of T to be the set 

Im T = {T(x) e V: X e U} 
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and the kernel of T to be the set 

Ker T = {x e U: T(x) = 0} . 

(Many authors call Im T the range of T, but we use this term to mean the 
space V in which T takes its values.) Since T(0) = £ V, we see that £ 
Im T, and hence Im T ^ 0. Now suppose x', y' £ Im T. Then there exist x, 
y £ U such that T(x) = x' and T(y) = y'. Then for any a, b £ ^ we have 

ax' + by' = aT(x) + bT(y) = T(ax + by) E Im T 

(since ax + by E U), and thus Im T is a subspace of V. Similarly, we see that 
e Ker T, and if x, y e Ker T then 

T(ax + by) = aT(x) + bT(y) = 

so that Ker T is also a subspace of U. Ker T is frequently called the null space 
ofT. 

We now restate Theorem 2.5 in our current terminology. 

Theorem 5.5 A linear transformation T E L(U, V) is an isomorphism if and 
only ifKerT = {0}. 

For example, the projection mapping T defined in Example 5.1 is not an 
isomorphism because T(0, 0, z) = (0, 0, 0) for all (0, 0, z) E IR^. In fact, if Xq 
and yo are fixed, then we have T(xo, yo, z) = (xq, yo, 0) independently of z. 

If T e L(U, V), we define the rank of T to be the number 

r(T) = dim(ImT) 

and the nullity of T to be the number 

nul T = dim(Ker T) . 

We will shortly show that this definition of rank is essentially the same as our 
previous definition of the rank of a matrix. The relationship between r(T) and 
nul T is given in the following important result. 

Theorem 5.6 If U and V are finite-dimensional over ^ and T E L(U, V), then 



r(T) + nul T = 



dimU 
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Proof Let {ui, . . . , Un} be a basis for U and suppose that Ker T = {0}. Then 
for any x £ U we have 

T(x) = T(i:XiUi) = i:XiT(Ui) 

for some set of scalars Xj, and therefore {T(Ui)} spans Im T. If 2ciT(Ui) = 0, 
then 

= 2ciT(Ui) = 2T(CiUi) = T(2ciUi) 

which implies that 2ciUi = (since Ker T = {0}). But the Ui are linearly inde- 
pendent so that we must have Ci = for every i, and hence {T(Ui)} is linearly 
independent. Since nul T = dim(Ker T) = and r(T) = dim(Im T) = n = dim U, 
we see that r(T) + nul T = dim U. 

Now suppose that Ker T ^ {0}, and let {wi, . . . , Wj,} be a basis for Ker T. 
By Theorem 2.10, we may extend this to a basis {w,, . . . , Wn} for U. Since 
T(Wi) = for each i = 1, . . . , k it follows that the vectors T(wk+i), . . . , T(Wn) 
span Im T. If 

for some set of scalars Ci, then 

0= |c/(>vp=|;r(c,.>vp = r(|;c,.wp 

j=k+l j=k+l j=k+l 

so that 2|= k+lCjWj E Ker T. This means that 

n k 

j=k+i j=i 
for some set of scalars ai. But this is just 

k n 
7=1 J=lc+l 

and hence 

El = • • • = ak = Ck+l = • • • = Cn = 

since the Wj are linearly independent. Therefore T(wk+i ),..., T(Wn) are lin- 
early independent and thus form a basis for Im T. We have therefore shown 
that 



dimU 



= k + (n - k) 



= dim(Ker T) + dim(Im T) 



= nul T + r(T) . I 
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The reader should carefully compare this theorem with Theorem 3.13 and 
Exercise 3.6.3. 

An extremely important special case of the space L(U, V) is the space 
L(V, V) of all linear transformations of V into itself. This space is frequently 
written as L(V), and its elements are usually called linear operators on V, or 
simply operators. 

Recall that Theorem 5.2 showed that the space L(U, V) is closed with 
respect to addition and scalar multiplication. Furthermore, in the particular 
case of L(V), the composition of two functions f , g G L(V) leads naturally to a 
"multiplication" defined by fg = f ° g E L(V). In view of Theorems 5.2 and 
5.3, it is now a simple matter to prove the following. 

Theorem 5.7 The space L(V) is an associative ring. 

Proof All that remains is to verify axioms (R7) and (R8) for a ring as given in 
Section 1.4. This is quite easy to do, and we leave it to the reader (see 
Exercise 5.2.1). I 

In fact, it is easy to see that L(V) is a ring with unit element. In particular, 
we define the identity mapping I E L(V) by I(x) = x for all x E V, and hence 
for any T E L(V) we have 

(IT)(x) = I(T(x)) = T(x) = T(I(x)) = (TI)(x) 

so that I commutes with every member of L(V). (However L(V) is certainly 
not a commutative ring in general if dim V > 1.) 

An associative ring A is said to be an algebra (or linear algebra) over jT 
if is a vector space over ^ such that 

a(ST) = (aS)T = S(aT) 

for all a E ^ and S, T G Jl. Another way to say this is that an algebra is a 
vector space on which an additional operation, called vector multiplication, 
is defined. This operation associates a new vector to each pair of vectors, and 
is associative, distributive with respect to addition, and obeys the rule a(ST) = 
(aS)T = S(aT) given above. Loosely put, an algebra is a vector space in which 
we can also multiply vectors to obtain a new vector. However note, for 
example, that the space IR^ with the usual "dot product" defined on it does not 
define an algebra because a • b is a scalar. Similarly, IR^ with the usual "cross 
product" is not an algebra because (a x b ) x c ;t a x (b x c). 
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Theorem 5.8 The space L(V) is an algebra over ^. 

Proof For any a E j/^, any S, T E L(V) and any x E V we have 

(a(ST))x = a(ST)(x) = aS(T(x)) = (aS)T(x) = ((aS)T)x 

and 

(a(ST))x = aS(T(x)) = S(aT(x)) = S((aT)x) = (S(aT))x . 

This shows that a(ST) = (aS)T = S(aT) and, together with Theorem 5.7, 
proves the theorem. I 

A linear transformation T E L(U, V) is said to be invertible if there exists 
a linear transformation T"' E L(V, U) such that TT"' = T'^T = I (note that 
technically TT"' is the identity on V and T"'T is the identity on U). This is 
exactly the same definition we had in Section 3.7 for matrices. The unique 
mapping T"' is called the inverse of T. 

Theorem 5.9 A linear transformation T E L(U, V) is invertible if and only if 
it is a bijection (i.e., one-to-one and onto). 

Proof First suppose that T is invertible. If T(xi) = T(x2) for Xi, Xj E U, then 
the fact that T"'T = I implies 

xi = T-iT(Xi) = T-1T(X2) = X, 

and hence T is injective. If y E V, then using TT"' = I we have 

y = I(y) = (TT-')y = T(T-Hy)) 

so that y = T(x) where x = T"'(y). This shows that T is also surjective, and 
hence a bijection. 

Conversely, let T be a bijection. We must define a linear transformation 
T"' E L(V, U) with the desired properties. Let y E V be arbitrary. Since T is 
surjective, there exists a vector x E U such that T(x) = y. The vector x is 
unique because T is injective. We may therefore define a mapping T"' : V ^ 
U by the rule T"'(y) = x where y = T(x). To show that T"' is linear, let yi, y2 E 
V be arbitrary and choose Xi, Xj E U such that T(xi) = yi and T(x2) = yi- 
Using the linearity of T we then see that 



and hence 



T(xi + X2) = yi + y2 
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T"Hyi + yi) = xi + X2 . 

But then 

T-'(yi + y2) = X1 + X2 = T-'(yi) + T-'(y2) . 

Similarly, if T(x) = y and a e J, then T(ax) = aT(x) = ay so that 

T-'(ay) = ax = aT'^y) . 

We have thus shown that T"' G L(V, U). Finally, we note that for any y G V 
and X G U such that T(x) = y we have 

TT-i(y) = T(x) = y 

and 

T-'T(x) = T-^y) = X 

so that TT ' =T iT = I. I 

A linear transformation T G L(U, V) is said to be nonsingular if Ker T = 
{0}. In other words, T is nonsingular if it is one-to-one (Theorem 5.5). As we 
might expect, T is said to be singular if it is not nonsingular. In other words, 
T is singular if Ker T ^ {0}. 

Now suppose U and V are both finite-dimensional and dim U = dim V. If 
Ker T = {0}, then nul T = and Theorem 5.6 shows that dim U = dim(Im T). 
In other words, we must have ImT = V, and hence T is surjective. 
Conversely, if T is surjective then we are forced to conclude that nul T = 0, 
and thus T is also injective. Hence a linear transformation between two finite- 
dimensional vector spaces of the same dimension is one-to-one if and only if 
it is onto. Combining this discussion with Theorem 5.9, we obtain the 
following result and its obvious corollary. 

Theorem 5.10 Let U and V be finite-dimensional vector spaces such that 
dim U = dim V. Then the following statements are equivalent for any linear 
transformation T G L(U, V): 

(a) T is invertible. 

(b) T is nonsingular. 

(c) T is surjective. 

Corollary A linear operator T G L(V) on a finite-dimensional vector space is 
invertible if and only if it is nonsingular. 
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Example 5.8 Let V = so that any x £ V may be written in terms of com- 
ponents as X = (xi, . . . , Xn). Given any matrix A = (ajj) £ MmxnCD, we define 
a linear transformation T : ->^'^ by T(x) = y which is again given in 
component form by 

n 

yi^^^ij^j ' i = l, ... , m . 

7=1 

We claim that T is one-to-one if and only if the homogeneous system 

UijXj = , i = \, ... , m 

7=1 

has only the trivial solution. (Note that if T is one-to-one, this is the same as 
requiring that the solution of the nonhomogeneous system be unique. It also 
follows from Corollary 5 of Theorem 3.21 that if T is one-to-one, then A is 
nonsingular.) 

First let T be one-to-one. Clearly T(0) = 0, and if v = (vi, . . . , Vn) is a 
solution of the homogeneous system, then T(v) = 0. But if T is one-to-one, 
then V = is the only solution. Conversely, let the homogeneous system have 
only the trivial solution. If T(u) = T(v), then 

= T(u) - T(v) = T(u - v) 

which implies that u-v = Ooru = v. / 

Example 5.9 Let T e L(R2) be defined by 

T(x,y) = (y, 2x-y) . 

If T(x, y) = (0, 0), then we must have x = y = 0, and hence Ker T = {0}. By 
the corollary to Theorem 5.10, T is invertible, and we now show how to find 

Suppose we write (x', y') = T(x, y) = (y, 2x - y). Then y = x' and 2x - y = 
y' so that solving for x and y in terms of x' and y' we obtain x = (l/2)(x' + y') 
and y = x'. We therefore see that 

T-i(x',y') = (x72 + y72,x') . 

Note this also shows that T is surjective since for any (x', y') El IR^ we found a 
point (x, y) = (x72 + y72, x') such that T(x, y) = (x', y'). / 

Our next example shows the importance of finite-dimensionality in 
Theorem 5.10. 
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Example 5.10 Let V = ^[x], the (infinite-dimensional) space of all polyno- 
mials over ^ (see Example 2.2). For any v £ V with v = 2'}= oSiX^ we define 
T e L(V) by 

n 

i=\ 

(this is just a "multiplication by x" operation). We leave it to the reader to 
show that T is linear and nonsingular (see Exercise 5.2.2). However, it is clear 
that T can not be surjective (for example, T takes scalars into polynomials of 
degree 1), so it can not be invertible. However, it is nevertheless possible to 
find a left inverse Tl"' for T. To see this, we let Tl"' be the operation of sub- 
tracting the constant term and then dividing by x: 

n 

i=l 

We again leave it to the reader (Exercise 5.2.2) to show that this is a linear 
transformation, and that Tl"'T = I while TTl"' I. 

While the above operation T is an example of a nonsingular linear trans- 
formation that is not surjective, we can also give an example of a linear trans- 
formation on ^[x] that is surjective but not nonsingular. To see this, consider 
the operation D = d/dx that takes the derivative of every polynomial in ^[x]. It 
is easy to see that D is a linear transformation, but D can not possibly be 
nonsingular since the derivative of any constant polynomial p(x) = c is zero. 
Note though, that the image of D is all of ^[x], and it is in fact possible to find 

a right inverse of D. Indeed, if we let Djf^if) = f^f(t) dt be the (indefinite) 

J 

integral operator, then 




and hence DDr ' = I. However, it is obvious that Dr 'D ;t I because Dr 'D 
applied to a constant polynomial yields zero. / 

Exercises 

1 . Finish the proof of Theorem 5.7. 

2. (a) Verify that the mapping A in Example 5.8 is linear. 

(b) Verify that the mapping T in Example 5.9 is linear. 

(c) Verify that the mapping T in Example 5.10 is linear and nonsingular. 

(d) Verify that T Tl"' ^ I in Example 5. 10. 
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3. Find a linear transformation T: IR^ ^ IR"^ whose image is generated by the 
vectors (1, 2, 0, -4) and (2, 0, -1, -3). 

4. For each of the following linear transformations T, find the dimension 
and a basis for Im T and Ker T: 

(a) T: defined by T(x, y, z) = (x + 2y - z, y + z, x + y - 2z). 

(b) T: 1R4 ^ |r3 defined by 

T(x, y, z, t) = (x - y + z + 1, X + 2z - 1, X + y + 3z - 3t) . 

5. Consider the space M2(IR) of real 2x2 matrices, and define the matrix 



Find the dimension and exhibit a specific basis for the kernel of the linear 
transformation T: M2(R) ^ M2(R) defined by T(A) = AB - BA = [A, B]. 

6. Let T: U — * V be a linear transformation with kernel Kj. If T(u) = v, 
show that T''(v) is just the coset u + Kj = {u + k: k £ Kj} (see Section 



7. Show that a linear transformation is nonsingular if and only if it takes lin- 
early independent sets into linearly independent sets. 

8. Consider the operator T: R^ defined by 



(a) Show that T is invertible. 

(b) Find a formula for T"'. 

9. Let E be a projection (or idempotent) operator on a space V, i.e., E-^ = E 
on V. Define U = Im E and W = Ker E. Show that: 

(a) E(u) = u for every u G U. 

(b) IfE^l, then E is singular. 

(c) V = U©W. 




1.5). 



T(x, y, z) = (2x, 4x - y, 2x + 3y - z) . 
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10. If S: U ^ V and T: V -» U are nonsingular linear transformations, show 
that S T is nonsingular. What can be said if S and/or T is singular? 

11. Let S: U V and T: V ^ W be linear transformations. 

(a) Show that T S: U ^ W is linear. 

(b) Show that r(T S) < r(T) and r(T S) < r(S), i.e., r(TS) < min{r(T), 
r(S)}. 

12. If S, T E L(V) and S is nonsingular, show that r(ST) = r(TS) = r(T). 

13. If S, T e L(U, V), show that r(S + T) < r(S) + r(T). Give an example of 
two nonzero linear transformations S, T E L(U,V) such that r(S + T) = 
r(S) + r(T). 

14. Suppose that V = U ® W and consider the linear operators E, and E2 on 
V defined by Ei(v) = u and E2(v) = w where u G U, w G W and v = u + 
w. Show that: 

(a) El and E2 are projection operators on V. 

(b) El + E2 = I. 

(c) E,E2 = = E2E1. 

(d) V = ImEi©ImE2. 

15. Prove that the nonsingular elements in L(V) form a group. 

16. Recall that an operator T G L(V) is said to be nilpotent if T° = for 
some positive integer n. Suppose that T is nilpotent and T(x) = ax for 
some nonzero x G V and some a G ^. Show that a = 0. 

17. If dim V = 1, show that L(V) is isomorphic to f. 

18. Let V = have the standard basis {e;}, and let T G L(V) be defined by 
T(ei) = (1, 0, 0, T(e2) = (0, 1, 1) and T(e3) = 1, 0). Is T invertible? 

19. Let V be finite-dimensional, and suppose T G L(V) has the property that 
r(T2) = r(T). Show that (Im T) D (Ker T) = {0}. 

5.3 MATRIX REPRESENTATIONS 

By now it should be apparent that there seems to be a definite similarity 
between Theorems 5.6 and 3.13. This is indeed the case, but to formulate this 
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relationship precisely, we must first describe the representation of a linear 

transformation by matrices. 

Consider a linear transformation T G L(U, V), and let U and V have bases 
{Ui, . . . , Un} and {vi, . . . , Vm } respectively. Since T(Ui) G V, it follows from 
Corollary 2 of Theorem 2.4 that there exists a unique set of scalars a^, . . . , 
ami such that 



ji 

7=1 

for each i = 1, . . . , n. Thus, the linear transformation T leads in a natural way 
to a matrix (aij) defined with respect to the given bases. On the other hand, if 
we are given a matrix (aij), then 2^= iVjaji is a vector in V for each i = 1, . . . , 
n. Hence, by Theorem 5.1, there exists a unique linear transformation T 
defined by T(Ui) = 2^ = iVjaji. 

Now let X be any vector in U. Then x = 2F= iXiUi so that 

/ n \ 

T(x) = T 



\<=i / 



i=l i=l j=\ 



But T(x) G V so we may write 

m 

Since {vj} is a basis for V, comparing these last two equations shows that 

n 

for each j = 1, . . . , m. The reader should note which index is summed over in 
this expression for yj. 

If we write out both of the systems T(Ui) = 2Y=iVjaji and yj = 2F=iajiXi, 
we have 

r(Mi) = aiiVi+ •••+a^iv^ 

; (1) 

and 

; (2) 
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We thus see that the matrix of coefficients in (1) is the transpose of the matrix 
of coefficients in (2). We shall call the m x n matrix of coefficients in equa- 
tions (2) the matrix representation of the linear transformation T, and we say 
that T is represented by the matrix A = (aij) with respect to the given 
(ordered) bases {ui} and {vi}. 

We will sometimes use the notation [A] to denote the matrix correspond- 
ing to an operator A £ L(U, V). This will avoid the confusion that may arise 
when the same letter is used to denote both the transformation and its repre- 
sentation matrix. In addition, if the particular bases chosen are important, then 
we will write the matrix representation of the above transformation as [A]u, 
and if A G L(V), then we write simply [A]v. 

In order to make these definitions somewhat more transparent, let us make 
the following observation. If x G U has coordinates (xi, . . . , Xn) relative to a 
basis for U, and y G V has coordinates (yi, . . . , ym) relative to a basis for V, 
then the expression y = A(x) may be written in matrix form as Y = [A]X 
where both X and Y are column vectors. In other words, [A]X is the coordi- 
nate vector corresponding to the result of the transformation A acting on the 
vector X. An equivalent way of writing this in a way that emphasizes the bases 
involved is 

[y]v = [A(x)]v = [AiSMu . 
If {Vj} is a basis for V, then we may clearly write 

Vi = Vj 6ji 

where the 8ji are now to be interpreted as the components of Vj with respect to 
the basis {vj}. In other words, v, has components (1,0,..., 0), V2 has com- 
ponents (0, 1, ... , 0) and so forth. Hence, writing out [A(ui)]v = 2f=iVjaji , 
we see that 





(a \ 
"11 










f ^ 1 




(a \ 







+ 


^21 


+ • 


• + 











.0. 




.0. 













so that [A(ui)] V is just the first column of [A]u . Similarly, it is easy to see that 
in general, [A(Ui)]v is the ith column of [A]u . In other words, the matrix rep- 
resentation [A]u of a linear transformation A G L(U, V) has columns that are 
nothing more than the images under A of the basis vectors of IJ. 

We summarize this very important discussion as a theorem for easy refer- 
ence. 
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Theorem 5.11 Let U and V have bases {ui, . . . , Un} and {vi, . . . , Vm} 
respectively. Then for any A £ L(U, V) the vector 

m 

is the ith column of the matrix [A]u = (ajj) that represents A relative to the 
given bases. 

Example 5.11 Let V have a basis {vi, V2, V3}, and let A E L(V) be defined by 

A(Vi) = 3vi +V3 
A(V2)= V1-2V2-V3 

Then the representation of A (relative to this basis) is 



/3 1 
-2 1 



. / 



The reader may be wondering why we wrote A(Ui) = SjVjaji rather than 
A(Ui) = 2jaijVj . The reason is that we want the matrix corresponding to a 
combination of linear transformations to be the product of the individual 
matrix representations taken in the same order. (The argument that follows is 
based on what we learned in Chapter 3 about matrix multiplication, even 
though technically we have not yet defined this operation within the frame- 
work of our current discussion. In fact, our present formulation can be taken 
as the definition of matrix multiplication.) 

To see what this means, suppose A, B E L(V). If we had written (note the 
order of subscripts) A(Vi) = SjayVj and B(Vi) = ^jbyVj, then we would have 
found that 

(A5)(v,) = A(5(v,)) = A(2/^.vp = ^jb^^A{v^) 

where c-,^ = Sjbijajk. As a matrix product, we would then have [C] = [B][A]. 
However, if we write (as we did) A(Vi) = SjVjaji and B(Vi) = SjVjbji, then we 
obtain 

(Afi)(v,) = A(fi(v,)) = AOljVjbj,) = ^jA(vj)bj, 
= ^j, k^k^kpji = ^k^k^ki 
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where now Cki = Sjatjbji. Since the matrix notation for this is [C] = [A][B], we 
see that the order of the matrix representation of transformations is preserved 
as desired. We have therefore proven the following result. 

Theorem 5.12 For any operators A, B e L(V) we have [AB] = [A][B]. 

From equations (2) above, we see that any nonhomogeneous system of m 
linear equations in n unknowns defines an m x n matrix (aij). According to our 
discussion, this matrix should also define a linear transformation in a consis- 
tent manner. 

Example 5.12 Consider the space IR^ with the standard basis 




so that any X e 



may be written as 




Suppose we have the system of equations 



which we may write in matrix form as [A]X = Y where 




Hence we have a linear transformation A(x) = [A]X. In particular. 
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We now see that letting the ith column of [A] be A(ei), we arrive back at the 
original form [A] that represents the linear transformation A(ei) = 2ei + and 
A(e2) = -ei + 3e2. / 

Example 5.13 Consider the space V = IR with basis vectors Vi = (1, 1) and 
V2 = (-1, 0). Let T be the linear operator on defined by 

T(x,y) = (4x-2y,2x + y) . 

To find the matrix of T relative to the given basis, all we do is compute the 
effect of T on each basis vector: 

T(vO = T(l, 1) = (2,3) = 3vi + V2 

T(V2) = T(-1,0) = (-4,-2) = -2vi + 2v2 . 

Since the matrix of T has columns given by the image of each basis vector, we 
must have 




. / 



Theorem 5.13 Let U and V be vector spaces over ^ with bases {ui, . . . , Un} 
and {vi, . . . , Vm} respectively. Suppose A £ L(U, V) and let [A] be the 
matrix representation of A with respect to the given bases. Then the mapping 
(|): A ^ [A] is an isomorphism of L(U, V) onto the vector space MmxnClF) of 
all m X n matrices over ^f. 



Proof Part of this was proved in the discussion above, but for ease of refer- 
ence, we repeat it here. Given any (a^) E MmxnC^O, we define the linear 
transformation A E L(U, V) by 

m 

7=1 

for each i = 1, . . . , n. According to Theorem 5.1, the transformation A is 
uniquely defined and is in L(U, V). By definition, [A] = (ay), and hence (|) is 
surjective. On the other hand, given any A E L(U, V), it follows from 
Corollary 2 of Theorem 2.4 that for each i = 1, . . . , n there exists a unique set 
of scalars a,i, . . . , ami E IF such that A(Ui) = 2'j^=iVjaji . Therefore, any A G 
L(U, V) has lead to a unique matrix (ajj) G Mmxn(^- Combined with the pre- 
vious result that (j) is surjective, this shows that (j) is injective and hence a 
bijection. Another way to see this is to note that if we also have B G L(U, V) 
with [B] = [A], then 
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{B - A)(M,) = B{u,)- A(M,) = 2 v/^,- - = . 

Since B - A is linear (Theorem 5.3), it follows that (B - A)x = for all x e U, 
and hence B = A so that ^ is one-to-one. 

Finally, to show that (j) is an isomorphism we must show that it is also a 
vector space homomorphism (i.e., a linear transformation). But this is easy if 
we simply observe that 

(A + B)(Ui) = A(Ui) + B(Ui) = SjVjRj, + 2jVjbj, = 2jVj(aj, + bji) 

and, for any c £ ^, 

(cA)(uO = c(A(uO) = c(2jVj ^0 = 2jVj(ci^0 . 

Therefore we have shown that 

[A + B] = [A] + [B] 

and 

[cA] = c[A] 

so that (|) is a homomorphism. I 

It may be worth recalling that the space MmxnClD is clearly of dimension 
mn since, for example, we have 

[" %c{' \,{' '\ . 

\c dj [0 0) [0 0) [l 0) [0 1) 

Therefore Theorem 5.13 provides another proof that dim L(U, V) = mn. 

Let us return again to the space L(V) = L(V, V) where dim V = n. In this 
case, each linear operator A £ L(V) will be represented by an n x n matrix, 
and we then see that the space MnClT) = Mnxn(^ of all n x n matrices over !f 
is closed under addition, multiplication, and scalar multiplication. By 
Theorem 5.13, L(V) is isomorphic to Mn(^, and this isomorphism preserves 
addition and scalar multiplication. Furthermore, it also preserves the 
multiplication of operators since this was the motivation behind how we 
defined matrix representations (and hence matrix multiplication). Finally, 
recall that the identity transformation I £ L(V) was defined by I(x) = x for all 
X £ V. In particular 



I(Ui) = 2jUj6ji 
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SO that the matrix representation of I is just the usual n x n identity matrix that 
commutes with every other n x n matrix. 

Theorem 5.14 The space MnCT) of all n x n matrices over ^ is a linear 
algebra. 

Proof Since Mn(^ is isomorphic to L(V) where dim V = n, this theorem 
follows directly from Theorem 5.8. I 

We now return to the relationship between Theorems 5.6 and 3.13. In par- 
ticular, we would like to know how the rank of a linear transformation is 
related to the rank of a matrix. The answer was essentially given in Theorem 
5.11. 

Theorem 5.15 If A E L(U, V) is represented by [A] = (aji) E MmxnC^O, then 
r(A)=r([A]). 

Proof Recall that r(A) = dim(Im A) and r([A]) = cr([A]). For any x E U we 
have 

A(x) = A(2xiUi) = 2xiA(uO 

so that the A(Ui) span Im A. But [A(Ui)] is just the ith column of [A], and 
hence the [A(Ui)] also span the column space of [A]. Therefore the number of 
linearly independent columns of [A] is the same as the number of linearly 
independent vectors in the image of A (see Exercise 5.3.1). This means that 
r(A) = cr([A]) = r([A]). ■ 

Suppose that we have a system of n linear equations in n unknowns 
written in matrix form as [A]X = Y where [A] is the matrix representation of 
the corresponding linear transformation A E L(V), and dim V = n. If we are to 
solve this for a unique X, then [A] must be of rank n (Theorem 3.16). Hence 
r(A) = n also so that nul A = dim(Ker A) = by Theorem 5.6. But this means 
that Ker A = {0} and thus A is nonsingular. Note also that Theorem 3.13 now 
says that the dimension of the solution space is zero (which it must be for the 
solution to be unique) which agrees with Ker A = {0}. 

All of this merely shows the various interrelationships between the matrix 
nomenclature and the concept of a linear transformation that should be 
expected in view of Theorem 5.13. Our discussion is summarized by the fol- 
lowing useful characterization. 
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Theorem 5.16 A linear transformation A e L(V) is nonsingular if and only 
if det[A] ;tO. 

Proof Let dim V = n. If A is nonsingular then nul A = 0, and hence r([A]) = 
r(A) = n (Theorem 5.6) so that [A]"' exists (Theorem 3.21). But this means 
that det [A] ^ (Theorem 4.6). The converse follows by an exact reversal of 
the argument. I 



Exercises 

1. Suppose A £ L(U, V) and let {ui}, {vi} be bases for U and V respec- 
tively. Show directly that {A(ui)} is linearly independent if and only if 
the columns of [A] are also linearly independent. 

2. Let V be the space of all real polynomials of degree less than or equal to 
3. In other words, elements of V are of the form f(x) = ao + aiX + a2X + 
asx^ where each ai G R. 

(a) Show that the derivative mapping D = d/dx is an element of L(V). 

(b) Find the matrix of D relative to the ordered basis {fj} for V defined 
by fi(x) = x^ -1 . 

3. Let T: IR^ be defined by T(x, y, z) = (x + y, 2z - x). 

(a) Find the matrix of T relative to the standard bases for IR^ and R^. 

(b) Find the matrix of T relative to the basis {a(} for R^ and {pi} for R^ 
where a, = (1, 0, -1), = (1, 1, 1), ag = (1, 0, 0), pi = (0, 1) and = (1, 
0). 

4. Relative to the standard basis, let T G L(R^) have the matrix representa- 
tion 

/ 1 2 1\ 
Oil. 

.-1 3 4^ 

Find a basis for Im T and Ker T. 

5. Let T G L(r3) be defined by T(x, y, z) = (3x + z, -2x + y, -x + 2y + 4z). 

(a) Find the matrix of T relative to the standard basis for R^. 

(b) Find the matrix of T relative to the basis {ai} given by «! = (1, 0, 1), 
a2 = (-1, 2, 1) and as = (2, 1, 1). 
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(c) Show that T is invertible, and give a formula for T ^ similar to that 
given in part (a) for T. 

6. Let T: j7^" -^j^^ be the linear transformation defined by 



/ n 



r(xi,...,x„) = 



\(=1 



(=1 / 



(a) Show that the matrix of T relative to the standard bases of and jT"^ 
is given by 



^1 1 ^1 2 



^21 ^22 



*2n 



mn / 



(b) Find the matrix representation of T: IR defined by 
T(x,y, z,t) = (3x-4y + 2z-5t, 5x + 7y-z-2t) 

relative to the standard bases of IR°. 

7. Suppose that T G L(U, V) has rank r. Prove that there exists a basis for U 
and a basis for V relative to which the matrix of T takes the form 



7, 0^ 

.0 0; 



[Hint: Show that Ker T has a basis {wi, . . . , Wm-r}, and then extend this 
to a basis {u,, . . . , Ur, w,, . . . , Wm-r} for U. Define Vj = T(Ui), and show 
that this is a basis for Im T. Now extend this to a basis for V.] 



8. Let {ei} be the standard basis for R^, and let {fi} be the standard basis for 

(a) Define T: R^ ^ R^ by T(ei) = T(e2) = fi and T(e3) = f i + fz- Write 
down the matrix [T]|. 

(b) Define S: R^ ^ R^ by S(fi) = (1, 2, 3) and S(f2) = (2, -1, 4). Write 
down [S]f. 

(c) Find ST(ei) for each i = 1, 2, 3, and write down the matrix [ST]e of 
the linear operator ST: R^ R^. Verify that [ST] = [S][T]. 
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Suppose T E L(V) and let W be a subspace of V. We say that W is 
invariant under T (or T-invariant) if T(W) C W. If dim W = m, show 
that T has a block matrix representation of the form 



(A B\ 
C 



where A is an m x m matrix. 



10. Let T e L(V), and suppose that V = U © W where both U and W are T- 
invariant (see the previous problem). If dim U = m and dim W = n, show 
that T has a matrix representation of the form 



(A 0\ 
^0 c 



where A is an m x m matrix and B is an n x n matrix. 



11. Show that A e L(V) is nonsingular implies [A"'] = [A]"' 



5.4 CHANGE OF BASIS 

Suppose we have a linear operator A G L(V). Then, given a basis for V, we 
can write down the corresponding matrix [A]. If we change to a new basis for 
V, then we will have a new representation for A. We now investigate the rela- 
tionship between the matrix representations of A in each of these bases. 

Given a vector space V, let us consider two arbitrary bases {ei, . . . , en} 
and { Bi, . . . , Bn} for V. Then any vector x E V may be written as either x = 
SxiCi or as X = SxiCi . (It is important to realize that vectors and linear 
transformations exist independently of the coordinate system used to describe 
them, and their components may vary from one coordinate system to another.) 
Since each Bi is a vector in V, we may write its components in terms of the 
basis {ei}. In other words, we define the transition matrix [P] = (pij) E 
M„(^by 

n 

for each i = 1, . . . , n. The matrix [P] must be unique for the given bases 
according to Corollary 2 of Theorem 2.4. 

Note that [P] defines a linear transformation P G L(V) by P(ei) = Bi. Since 
{P(ei)} = {Bi} spans Im P and the Bi are linearly independent, it follows that 
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r(P) = n so that P is nonsingular and hence P"^ exists. By Theorem 5.13, we 
conclude that [P"^] = [P]"^ (However, it is also quite simple to show directly 
that if a linear operator A is nonsingular, then [A"'] = [A]"'. See Exercise 
5.3.11). 

Let us emphasize an earlier remark. From Theorem 5.11, we know that 
[Bi] = [P(ei)] is just the ith column vector of [P]. Since relative to the basis {ei} 
we have ei = (1, 0, . . . , 0), ej = (0, 1, . . . , 0) and so on, it follows that the ith 
column of [P] represents the components of Bi relative to the basis {Ci}. In 
other words, the matrix entry pji is the jth component of the ith basis vector Bi 
relative to the basis {Ci}. 

The transition matrix enables us to easily relate the components of any x £ 
V between the two coordinate systems. To see this, we observe that 

X = 2i Xiei = 2j Xj-ej = 2i, j Xj-eiPij = 2i, j PijXjei 

and hence the uniqueness of the expansion implies Xi = SjPijXj so that 

Xj — SiP jiXi . 

This discussion proves the following theorem. 

Theorem 5.17 Let [P] be the transition matrix from a basis {Ci} to a basis 
{Bi} for a space V. Then for any x G V we have 

Me = [P]-Hx]e 

which we sometimes write simply as X = P"'X. 

From now on we will omit the brackets on matrix representations unless 
they are needed for clarity. Thus we will usually write both a linear transfor- 
mation A G L(U, V) and its representation [A] G MmxnCn as simply A. 
Furthermore, to avoid possible ambiguity, we will sometimes denote a linear 
transformation by T, and its corresponding matrix representation by A = (ay). 

Using the above results, it is now an easy matter for us to relate the repre- 
sentation of a linear operator A G L(V) in one basis to its representation in 
another basis. If A(ei) = SjCjaji and A(Bi) = 2jBjaji , then on the one hand we 
have 

A(Bi) = 2j Bj-aji = 2j, kCkPkj-a-ji 
while on the other hand. 



A(ei) = A(2j CjPji) = 2j A(ej)pji = 2j, k etakjPji 
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Therefore, since {e^} is a basis for V, we may equate each component in these 
two equations to obtain 2j ptjaji = 2j akjPji or 

ari = 2j,kP 'rk^kjPji • 

In matrix notation, this is just (omitting the brackets on P) 

[A]e = P-'[A]eP 

which we will usually write in the form A = P"' AP for simplicity. 

If A, B e MnC^O^ then B is said to be similar to A if there exists a nonsin- 
gular matrix S such that B = S"'AS, in which case A and B are said to be 
related by a similarity transformation. We leave it to the reader to show that 
this defines an equivalence relation on Mn(^ (see Exercise 5.4.1). 

Since we have shown that in two different bases a linear operator A is rep- 
resented by two similar matrices, we might wonder whether or not there are 
any other matrices representing A that are not similar to the others. The 
answer is given by the following. 

Theorem 5.18 If T G L(V) is represented by A relative to the basis {Ci}, then 
a matrix A G Mn(i?0 represents T relative to some basis if and only if A is 
similar to A. If this is the case, then 

A = P-'AP 

where P is the transition matrix from the basis {ei} to the basis {Bi}. 

Proof The discussion above showed that if A and A represent T in two dif- 
ferent bases, then A = P"^AP where P is the transition matrix from {ei} to {ei}. 

On the other hand, suppose that T is represented by A in the basis {ei}, 
and assume that A is similar to A. Then A = P"^AP for some nonsingular 
matrix P = (pij). We define a new basis {Bi} for V by 

Bi = P(ei) = 2j ejPji 

(where we use the same symbol for both the operator P and its matrix repre- 
sentation). Then 



T(ei) = T(2j ejPji) = 2j T(ej)pji = 2j, k etakj-Pji 
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while on the other hand, if T is represented by some matrix C = (cji) in the 
basis {Bi}, then 

T(ei) = 2j CjCji = 2j, kekPkjCji . 
Equating the coefficients of e^ in both of these expressions yields 

2j fikj-Pji = 2j PkjCji 

SO that 

Cri = 2j,kP''rkakjPji 

and hence 

C = P-'AP = A . 

Therefore A represents T in the basis {Bi}. I 

Note that by Theorem 4.8 and its corollary we have 

detA = det(P-'AP) = (det p-')(det A)(det P) = detA 

and hence all matrices which represent a linear operator T have the same 
determinant. Another way of stating this is to say that the determinant is 
invariant under a similarity transformation. We thus define the determinant 
of a linear operator T G L(V) as det A, where A is any matrix representing 
T. 

Another important quantity associated with a matrix A G MnClT) is the 
sum 2f = 1 aii of its diagonal elements. This sum is called the trace, and is 
denoted by TrA (see Exercise 3.6.7). A simple but useful result is the 
following. 

Theorem 5.19 If A, B e MnCO, then Tr(AB) = Tr(BA). 

Proof We simply compute 

Tr(A5) = i:,(AB), = \ jU^bj^ = ^j^fijfiy = 2/5A)^ 
= Tr(5A) . I 

From this theorem it is easy to show that the trace is also invariant under a 
similarity transformation (see Exercise 4.2.14). Because of this, it also makes 
sense to speak of the trace of a linear operator. 
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Example 5.14 Consider the space V = IR^ with its standard basis ei = (1, 0) 
and 0,2 = (0, 1), and let Bi = (1, 2), 62 = (3, -1) be another basis. We then see 
that 

?i = + 2^2 
62 = 3ej — 62 

and consequently the transition matrix P from {ei} to {Bi} and its inverse P"' 
are given by 



'\ 3 
.2 -1 



and 



P-i = 



1/7 3/7^ 
\2I1 -1/7, 



Note that P ' may be found either using Theorem 4.11, or by solving for {ei} 
in terms of {Bi} to obtain 

ei=(l/7)?i+(2/7)i-2 
e2=(3/7)?i-(l/7)?2 

Now let T be the operator defined by 

r(ei) = (20/7M- (2/7)^2 
7(^2) = (-3/7M+ (15/7)^2 

so that relative to the basis {Ci} we have 

/20/7 -3/7\ 



A = 



^-2/7 15/7^ 



We thus find that 



A = P-^AP = 



'1/7 


3/7 \ 


'20/7 


-3/7 Vl 3' 




'2 0\ 


.2/7 


-1/vj 


-2/7 


15/7]^ -1, 




.0 3] 



Alternatively, we have 

T(ei) = T(ei + 262) = T(ei) + 2T(e2) = 2e, + 4e2 = 2ei 
T(e2) = T(3ei - e2) = 3T(ei) - T(e2) = (63/7)ei - 3e2 = 3e2 



so that again we find 



A = 



'1 0^ 

,0 3; 
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We now see that 

Tr A = 20/7 + 15/7 = 5 = Tr A 

and also 

detA = 6 = detA 

as they should. / 

We point out that in this example, A turns out to be a diagonal matrix. In 
this case the basis {ei} is said to diagonalize the operator T. While it is 
certainly not true that there always exists a basis in which every operator is 
diagonal, we will spend a considerable amount of time in Chapters 7 and 8 
investigating the various standard forms (called normal or canonical) that a 
matrix representation of an operator can take. 

Let us make one related additional comment about our last example. 
While it is true that (algebraically speaking) a linear operator is completely 
determined once its effect on a basis is known, there is no real geometric 
interpretation of this when the matrix representation of an operator is of the 
same form as A in Example 5.14. However, if the representation is diagonal as 
it is with A, then in this basis the operator represents a magnification factor in 
each direction. In other words, we see that A represents a multiplication of 
any vector in the Ci direction by 2, and a multiplication of any vector in the 62 
direction by 3. This is the physical interpretation that we will attach to eigen- 
values (see Chapter 7). 

Exercises 

1. Show that the set of similar matrices defines an equivalence relation on 

2. Let {Ci} be the standard basis for R^, and consider the basis fi = (1, 1, 1), 
f2 = (l, l,0)andf3 = (l,0, 0). 

(a) Find the transition matrix P from {ej} to {fj}. 

(b) Find the transition matrix Q from {fi} to {ei}. 

(c) Verify that Q = P-'. 

(d) Show that [v]f = P"' [v]e for any v G R^. 

(e) Define T E L(R^) by T(x, y, z) = (2y + z, x - 4y, 3x). Show that [T]f 
= P-'[T]eP. 

3. Let {ei, ^2} be a basis for V, and define T E L(V) by T(ei) = 3ei - 2e2 
and T(e2) = ei + 4e2. Define the basis {fi} for V by fi = ei + e2 and f2 = 
2ei + 3e2. Find [T]f. 
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Consider the field C as a vector space over IR, and define the linear 
"conjugation operator" T G L(C) by T(z) = z* for each z E C. 

(a) Find the matrix of T relative to the basis {cj} = {1, i}. 

(b) Find the matrix of T relative to the basis {f,} = {1 + z, 1 + 2i}. 

(c) Find the transition matrices P and Q that go from {cj} to {f|} and 
from {f|} to {ej} respectively. 

(d) Verify that Q = P-1. 

(e) Show that [T] f = [T] e P. 

(f ) Verify that Tr [T]f = Tr [T]e and det [T]f = det [T]e. 

Let {Ci}, {fi} and {gi} be bases for V, and let P and Q be the transition 
matrices from {ej} to {fi} and from {fj} to {gi} respectively. Show that 
PQ is the transition matrix from {ci} to {gi}. 



Let A be a 2 X 2 matrix such that only A is similar to itself. Show that A 
has the form 

'a 0\ 
^0 a) ' 



7. Show that similar matrices have the same rank. 

8. Let A, B and C be linear operators on IR^ with the following matrices 
relative to the standard basis {ei}: 



/ 4 6^ 






\ 


r2 -3, 



' 1/2 ->/3/2 
,V3/2 1/2 ) 



' 7 3^ 

-10 -^] 



(a) If f, = (2, -1) and U = (3, -2), show that A(fi) = fj and A(f2) = 0. 

(b) Find [A] f. 

(c) What is the geometric effect of A? 

(d) Show that B is a rotation about the origin of the xy-plane, and find 
the angle of rotation (see Example 1.2). 

(e) If fi = (1, -2) and U = (3, -5), find C(fi) and C(f2). 

(f) Find[C]f. 

(g) What is the geometric effect of C? 



9. 



(a) Let {Ci} be the standard basis for R", and let {fi} be any other ortho- 
normal basis (relative to the standard inner product). Show that the 
transition matrix P from {Cj} to {fj} is orthogonal, i.e., P^ = P"^ 
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(b) Let T 
basis: 



L(IR ) have the following matrix relative to the standard 

(2 Q\ 
4 
3 



Find the matrix of T relative to the basis fi = (2/3, 2/3, -1/3), f2 = 
(1/3, -2/3, -2/3) and fs = (2/3, -1/3, 2/3). 



10. Let T G L(IR^) have the following matrix relative to the standard basis 
{ei} for R^: 



c d 



(a) Suppose there exist two linearly independent vectors fi and fj in IR 
with the property that T(fi) = Xjfi and T(f2) = Xjfj (where h e R). If P is 
the transition matrix from the basis {ei} to the basis {fi}, show that 



[r],=p-i[r] p = 



A^, 



(b) Prove there exists a nonzero vector x 
T(x) = X if and only if 



IR with the property that 



a-l b 
c d-1 



= 



(c) Prove there exists a one-dimensional T-invariant subspace of IR^ if 
and only if 

a- A b 
c d-k 



= 



for some scalar X. (Recall that a subspace W is T-invariant if T(W) C W.) 

11. If 6 e IR, show that the matrices 

/cosS -sin0\ 
lsin0 cosSJ 



and 
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\ 



-ie 



are similar over the complex field. [Hint: Suppose T £ L(C^) has the first 
matrix as its representation relative to the standard basis. Find a new 
basis {vi, V2} such that T(vi) = exp(i0)vi and T(v2) = exp(-j0)v2.] 

12. Let V = R have basis vectors ei = (1, 1) and e2 = (1, -1). Suppose we 
define another basis for V by ei = (2, 4) and 62 = (3, 1). Define the tran- 
sition operator P E. L(V) as usual by Bi = Pej. Write down the matrix [P]|. 

13. Let U have bases {Ui} and {Ui} and let V have bases {vj and {Vj}. 
Define the transition operators P G L(U) and Q G L(V) by Ui = PUi and 
Vi = QVi. If T G L(U, V), express [T]^ in terms of [T]l 



14. Show that the transition matrix defined by the Gram-Schmidt process is 
upper-triangular with strictly positive determinant. 
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(see also Laplace expansion) 
Extended real number system 14 
Exterior of a set 703 
Exterior algebra 574 
Exterior product 563 
Exterior r-forms 554 



Factor group 64 
Factor theorem 262 
Factorial 18 
Field 54 

characteristic of 58 
Field of quotients 284 
Finite field 

characteristic of 291 

order of 288 
Finite-dimensional 77 
Formal linear combinations 578 
Fourier coefficients 106, 655 
Free variables 142 
Free vector space 579 
Function 4 
Functional 665 

Fundamental Theorem of Algebra 276, 693 
Fundamental Theorem of Arithmetic 29 



Gaussian elimination 124 
Generalized permutation symbol 565 
Gershgorin's theorem 307 
Graded associative algebra 575 
Graded components 574 
Gram-Schmidt process 109 
Gramian 197 
Grassmann algebra 574 
Greatest common divisor 26, 263 
Greatest lower bound 8 
Group 30 

abelian 30 

center of 67 

class 35 

conjugate elements of 35 
cyclic 291 
direct product 34 
finite 3 1 

homomorphism 49 
infinite 3 1 
multiplication 30 
order of 3 1 

representation 33, 335 
g- volume 615 
Hausdorff property 687 



Heine-Borel theorem 687, 691 
Hermitian adjoint 383 
Hermitian form 48 1 

associated quadratic form 482 

nonnegative semidefinite 482 

positive definite 482 
Hermitian inner product 620 
Hilbert basis 660 
Hilbert space 630, 640 
Homomorphic groups 49 
Homomorphism 49 

kernel of 50 

vector space 79 
Holder's inequality 642 
Hypcrcompanion matrix 420 
Hyperplane 457 
Hyperspace 457 

Ideal 59 

generated by 275 

unit 275 
Idempotent 157, 232, 353 
Identity mapping 6, 227 
Identity matrix 136 
Indeterminate 115, 255 
Index set 2 
Induction 17 
Infimum 8 
Infinite series 708 
Infinity symbols 14 
Injective 5 
Inner product 98 

Hermitian 98 

indefinite 609 

nondegenerate 451, 609 
Inner product space 99 

complex 99 

real 99 
Integers 

modulo n 57, 285 
Integral domain 57 
Interior of a set 702 
Interior product 572 
Invariant direct sum decomposition 332 
Invariant factors 398 
Invariant subspace 329 
Invariant volume element 617 
Irrational number 12 
Irreducible representation 335 
Isolated point 622 
Isometry 113,498 
Isomorphic 59, 79 
Isomorphic groups 50 
Isomorphism 50, 79 

inner product space 501 
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nullity 225 




orientation preserving 608 


Jacobian 599 


range 225 


Jointly continuous 648 


rank 225 


Jordan form 376, 422 


reducible 332 


uniqueness of 427 


restriction of 330 




singular 229 




volume preserving 608 


Kronecker delta 105 


Linearly dependent 75 


Kronecker product 581 


Linearly independent 75 




Lorentz frame 613 




Lorentz transformation 616 


Lagrange interpolation formula 280 


Lower bound 8 


Lagrange's theorem 61 


Lower limit 713 


Laplace expansion 573 


Lowering the index 610 


Law of cosines 96 




Law of inertia 478 




Least common multiple 29, 272 


Mapping 4 


Least upper bound 8 


alternating 180 


Least upper bound property 12 


associative 6 


Left identity 3 1 


bilinear 578 


Left inverse 31, 157 


commutative 6 


Left zero divisor 163 


composite 6 


Levi-Civita 


domain 4 


symbol 560 


image 4 


tensor 560 


inverse 5 


Limit 625 


inverse image 4 


Limit inferior 713 


multilinear 180, 544 


Limit point 700 


one-to-one 5 


Limit superior 713 


onto 5 


Linear extension theorem 639 


range 4 


Linear algebra 227 


restriction 4 


Linear combination 72 


Matrix 117 


Linear equations 115 


adjoint 183, 383 


coefficients of 1 1 5 


anticommutator 156, 184 


constant term 115 


antisymmetric 156, 184,469 


solution vector 115 


block 204 


system of 1 1 6 


canonical form 1 69 


equivalent 118 


classical adjoint 191 


homogeneous 138 


column rank 128 


nonhomogeneous 138 


columns 117 


nontrivial solution 138 


commutator 154, 156 


trivial solution 138 


conjugate transpose 183 


Linear functional 221 


derogatory 409 


Linear manifold 649 


diagonal 154, 165 


Linear operators 227 


direct product 581 


Linear span 72 


distinguished elements 124 


Linear transformation 79, 215 


equivalent 515 


diagonalizable 318 
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image 224 


Hprmitian '^8'? 489 
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inverse 228 


Hermitian adjoint 482 


invertible 228 


idempotent 157 


kernel 225 


inverse 157 


matrix representation of 235 


invertible 157 


negative of 219 


irreducible 427 


nonsingular 229 
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lower-triangular 156, 162, 177 
negative 148 
nilpotent 184 
nonderogatory 409 
nonsingular 157 
normal 383, 515 
orthogonal 183, 249, 384, 502 
product 148 
rank 136 

reduced row-echelon form 123 
reducible 333, 427 
row equivalent 123 
row rank 128 
row-echelon form 123 
row-reduced form 125 
rows 117 
singular 157 
size 117 

skew-Hermitian 388 
skewsymmetric 156 
square 148 
sum 147 

supertriangular 376 
symmetric 156, 184, 384, 470 
tensor product 581 
trace 155, 246 
transpose 152 
unit 392 

unitarily similar 515 

unitary 183, 383, 502 

upper-triangular 156, 162, 177 
Matrix exponential series 530 
Matrix of coefficients 116 
Maximal element 8 
Metric 104, 680 
Metric space 680 

complete 699 
Metric tensor 580 

contravariant 611 

covariant 610 

index of 613 

Lorentz 613 

Riemannian 613 
Metric volume form 615 
Minimal element 9 
Minimal polynomial 299, 313, 326 

of a vector 323 
Minkowski's inequality 643 
Minor 188 
Minor matrix 187 
Module 69 
Multilinear form 544 
Multiplicity 

algebraic 345 

geometric 345 



Natural mapping 453 
Natural numbers 2 
n-cell 689 
Neighborhood 690 

deleted 690 
Nilpotent 233 

index of nilpotency 369 

operator 306, 369 
Nondegenerate 462 
Nonnegative 2 
Norm 619 

Normal coordinates 489 
Normal matrix 515 
Normed vector space 101, 620 
Null space 154, 225 



1 -forms 447 
Open 

ball 681 

cover 686 

interval 14 

set 681 

subcover 686 
Operator 490, 653 

adjoint 673 

anti-Hermitian 508 

antisymmetric 511 

Hermitian 495, 676 

isometric 498, 499, 677 

nonnegative 508 

normal 509, 677 

orthogonal 499, 501 

positive 508 

positive definite 508 

positive semidefinite 508 

self-adjoint 495, 676 

skew-Hermitian 508 

symmetric 511 

unitary 499, 678 
Operator norm 637 
Order of a vector 440 
Ordered by inclusion 8 
Orientation 606, 608 

negative 592, 608 

positive 592, 608 
Oriented vector space 606, 608 
Oriented volume 592 
Orthogonal 102 

compliment 105, 620 

projection 217, 354 

set 105 
Orthonormal set 105 
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Parallelogram law 103, 621 
Parallelpiped 

base 589 

height 589 

r-dimensional 589 

r- volume 589 
Parseval's equation 662 
Partial isometry 507 
Partial ordering 7 

bounded above 8 

induced 8 

largest element 8 

smallest element 8 

upper bound 8 
Partially ordered set 7 
Partition 9 

induced 10 
Path 720 

Path connected 720 
Pauli spin matrices 156 
iP-elementary operations 390 
Permutation 35 

even 44 

odd 44 

one-line notation 36 

sign of 46 

two-line notation 35 
Permutation group 40 
cycle 41 
degree of 40 

equivalent elements in 41 

orbit 41 
Permutation symbol 560 
Perpendicular 102 
Polar decomposition 540 
Polar form identities 103 
Polygonally connected 721 
Polynomial 253 

associates 262 

coefficients of 253 

constant 256 

degree of 256 

factor of 261 

greatest common divisor 397 

irreducible 262 

leading coefficient of 256 

minimal300, 314, 327 

monic 256 

norm of 397 

reducible 262 

root 261, 297 

zero 261, 297 
Polynomial equation 261 

solution 261 
Polynomial function 255 



Positive integers 2 
Positive transformation 539 
Power set 7 
Preimage 4 

Primary decomposition theorem 339 
Prime number 25 
Prime polynomial 262 
Principal ideal 60 

generator of 60 
Product 625 

Projection 157, 232, 352, 654 
Pull-back 595 
Push-forward 602 
Pythagorean theorem 95, 103, 622 



Quadratic 

form 471 

diagonal representation 477 

polynomial 471 
Quaternions 280 
Quotient 25 
Quotient group 64 
Quotient ring 64 
Quotient space 362 



Raising an index 611 
Rank 

of a bilinear form 465 
of a matrix 135 
Ratio test 717 

Rational canonical form 416 
Rational numbers 2 
Rayleigh quotient 513 
Rearrangement lemma 34 
Reducible representation 333 
Reflexive space 671 
Relation 7 

Relatively prime 28, 263 
Remainder 25 
Remainder theorem 261 
Resolution of the identity 524 
r-forms 554 

Riesz representation theorem 666 
Riesz-Fischer theorem 663 
Right identity 31 
Right inverse 31, 157 
Right zero divisor 163 
Ring 53 

associates 262 

associative 53 

commutative 53 

embedded 282 

extension 282 

homomorphism 56 
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kernel of 59 
isomorphism 59 
with unit element 53 

Ring of sets 4 

r-linear form 544 

Root 261 

multiplicity of 281,305 

Root test 717 

Row canonical form 125 

Row space 128 

Row-column-equivalent 169 



Scalar 69 

Scalar mapping 301 
Scalar multiplication 68 
Scalar product 94 
Scalar triple product 588 
Schur canonical form 384 
Schur's lemma 335 
Schwartz's inequality 20 

generalized 649 
Second dual 222, 452 
Secular equation 309 
Separable 647, 695 
Sequence 696 

Cauchy 699 

decreasing 699 

increasing 699 

hmit of 622, 696 

monotonic 699 

range 697 
Series 708 

rearrangement of 710 
Sesquilinear form 620 
Set 2 

closed 683 

complement of 2 

countable 1 1 

countably infinite 1 1 

disjoint 3 

family of 2 

finite 1 1 

infinite 11 

intersection 3 

open 681 

symmetric difference 4 
uncountable 1 1 
union 2 
Shuffle 563 

Signature 477 

Signed permutation matrix 389 
Similar matrices 184, 245 
Similarity class 329 
Similarity invariants 408 
Similarity transformation 184, 245 



Simple root 305 

Smith canonical form 400 

Solution set 116 

Space of linear functionals 222 

Space of linear transformations 220 

Spectral decomposition 524 

Spectral theorem 525 

Spectrum 346 

degenerate 346 
Square root 15 
Standard basis 79 
Standard inner product 99, 620 
Standard orientation 608 
Subdeterminant 185 
Subgroup 33 

index of 62 

normal 62 
Submatrix 185, 193,209 
Subsequence 707 
Subsequential limit 707 
Subset 2 

proper 2 
Subspace 72, 649 

closed 649 

generated by 72, 660 

intersection of 86 

invariant 243, 329 

irreducible 518 

null 618 

of a metric space 684 

proper 72 

spacelike 618 

spanned by 72 

sum of 74, 86 

timelike 618 

trivial 72 
Summation convention 545 
Sup norm 625, 629, 633 
Superdiagonal 155, 370 
Superset 2 
Supremum 8 
Surjective 5 

Sylvester's theorem 478 
Symmetric group 37 
Symmetrizing mapping 556 



T-cyclic subspace 432 

generated by 432 
T-invariant subspace 243 
Tensor 545 

antisymmetric 553, 554 

classical law of transformation 550 

components 545, 547 

contraction 552 

contravariant order 545 
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covariant order 545 
rank 545 

skew-symmetric 553 

symmetric 553, 554 

trace 552 

type 545 
Tensor algebra 574 
Tensor product 462, 464, 547, 580 
Total 660 
Total ordering 8 
Trace 155 

Transcendental number 17 
Transition matrix 243 

orthogonal 249 
Transpose 

of a linear transformation 459 

of a matrix 153 
Transpositions 44 
Triangle inequality 101 
Triangular form theorem 367, 376 
Two-sided inverse 157 



Volume forms 607 
equivalent 607 



Wedge product 462, 563 
Well-defined 4 
Well-ordered 17 
Weyl's formula 536 



Zero divisor 57 
Zero mapping 219 
Zero matrix 148 
Zom's lemma 9 



Uniformly continuous 623 

Unique factorization theorem 266 

Unit (of a ring) 262 

Unit cube 593 

Unit matrix 392 

Unit vector 99 

Unitarily similar 385, 515 

Unitary 183, 383, 499, 502, 678 

Unitary space 99, 508 

Unknowns 115 

Upper limit 713 



Vandermonde matrix 195 
Vector 69 

length of 99 

lightlike 614 

norm of 99 

spacelike 614 

time like 614 
Vector multiplication 227 
Vector space 68 

complex 69 

dimension of 77, 83 

generated by 578 

infinite-dimensional 640 

isometric 113 

normed 101 

ordinary Euclidean 613 

pseudo-Euclidean 613 

real 69 

singular 613 
Vector space homomorphism 79 



